M2R: Micro-Macro Retrieval Framework

• Micro-Macro Retrieval (M2R) is a dual-stage framework that couples coarse macro retrieval during reasoning with on-demand micro retrieval for precise answer grounding.
• It addresses the proximity bottleneck by distilling key evidence during reasoning and reinserting it immediately before answer generation to enhance factual consistency.
• M2R employs a curriculum learning approach, training separate stages for evidence extraction and answer generation, which leads to improved accuracy and reduced hallucination.

Micro-Macro Retrieval (M2R) is a retrieve-while-generate framework for reducing hallucination in long-form LLM generation by combining two retrieval regimes: a macro level that retrieves coarse-grained evidence from external sources during reasoning, and a micro level that retrieves answer-aligned facts from an internal key-information repository immediately before answer generation (Feng et al., 10 Apr 2026). The framework is motivated by a specific failure mode of retrieval-augmented generation: external evidence can be retrieved and used during multi-step reasoning, yet key information may still end up far from the final output tokens. M2R addresses this key-information-to-output proximity bottleneck by distilling essential results during reasoning and reusing them at answer time, so that the final output is grounded in nearby evidence rather than distant, noisy context (Feng et al., 10 Apr 2026).

1. Problem setting and the proximity bottleneck

The motivating claim behind M2R is that hallucination becomes worse in long-form generation because two failure modes compound. First, redundant or long retrieved context can obscure evidence: important facts may be buried inside large passages and become difficult to identify and use correctly. Second, long reasoning chains can cause forgetting or drift: intermediate evidence and conclusions may no longer reliably constrain the final answer. The framework explicitly connects this behavior to the empirically observed “lost in the middle” or long-context effect, and its Appendix B gives a positional-encoding perspective using RoPE to argue that attention contribution declines with distance. In that formulation, evidence closer to the answer is more likely to be incorporated faithfully (Feng et al., 10 Apr 2026).

Within this problem formulation, standard retrieval-augmented LLMs are described as insufficient not because they fail to retrieve evidence, but because they do not guarantee that key information stays near the output tokens. M2R is therefore defined less as a retrieval expansion than as a proximity-control mechanism: external evidence is gathered during reasoning, compressed into answer-aligned entries, and then re-accessed at the point of answer generation (Feng et al., 10 Apr 2026).

A common misunderstanding is to treat M2R as a synonym for multi-turn retrieval. The framework is narrower and more structured than that. Its distinctive component is not merely repeated retrieval, but the separation between macro retrieval over external sources and micro retrieval over an internal repository $M$, with the latter occurring in the answer phase and serving direct grounding of the final output (Feng et al., 10 Apr 2026).

2. Dual-stage retrieval architecture

M2R is organized around a two-level retrieval design. At the macro level, retrieval occurs during the model’s > phase. The model performs one or more external retrieval calls tagged as <macro_tool_call>, and the returned passages provide coarse-grained evidence from external sources. When the model identifies answer-relevant facts during reasoning, it stores them into an internal key-information repository using <key_info_save>. The stored content is described as “answer-aligned evidence” or “essential results” in a key-information repository $M$ (Feng et al., 10 Apr 2026).

At the micro level, retrieval occurs during the <answer> phase. The model queries the internal repository with <micro_tool_call>, receives retrieved values in <micro_response>, and then generates the final answer grounded on those retrieved key facts. The framework’s central mechanism is that these values are inserted immediately before the answer tokens, thereby enforcing the desired proximity between evidence and output (Feng et al., 10 Apr 2026).

The procedural structure is explicit:

1. input question $x$;
2. enter <think> phase;
3. perform one or more macro retrievals from an external corpus;
4. reason over the retrieved passages;
5. store answer-relevant facts using <key_info_save>;
6. build repository $M$;
7. enter <answer> phase;
8. use <micro_tool_call> to retrieve saved key facts from $M$;
9. place retrieved facts immediately before answer tokens; 10. generate the final answer, with values boxed as required (Feng et al., 10 Apr 2026).

The framework formalizes this staging with a two-stage policy in which <think> with macro retrieval is followed by SaveKey, and <answer> is then conditioned on micro retrieval over $M$. In operational terms, macro retrieval gathers and distills evidence, while micro retrieval re-injects the distilled evidence at answer time. This design is intended to prevent forgetting of earlier intermediate results and to reduce hallucination by grounding the answer in recently retrieved, answer-aligned facts (Feng et al., 10 Apr 2026).

Repository construction is therefore not an auxiliary logging operation. It is the intermediate representation that links reasoning to answer generation. The saved content is expected to be directly relevant to the final answer, concise, and atomic enough to be queried later. A one-shot grounding variant that feeds all saved key information to the model at once during answering performs worse, which the paper interprets as evidence that precision timing of evidence insertion matters (Feng et al., 10 Apr 2026).

3. Optimization, masking, and curriculum learning

M2R is trained with reinforcement learning using GRPO and a curriculum schedule rather than a single joint optimization pass. The training objective is retrieval-conditioned: policy terms are evaluated with the outputs of both retrieval stages, and retrieval result masking is introduced because retrieval spans are produced by tools or the environment rather than by the policy itself. If $m_t \in \{0,1\}$ indicates whether token $t$ is policy-generated, the masked log-probability is

$$\log T_\theta(y|x)= \frac{\sum_t m_t \log T_\theta(y_t \mid y_{<t}, x)}{\max(1,\sum_t m_t)}.$$

This masking is used to preserve correct credit assignment and stabilize reinforcement learning (Feng et al., 10 Apr 2026).

The reward design is rule-based. A format reward checks whether a rollout obeys the required template, including valid <macro_tool_call> and <key_info_save> in reasoning, valid <micro_tool_call> in answering, and final answer values enclosed in \boxed{}. The answer reward is a weighted combination of three F1-based subrewards:

$$r_{\text{ans}} = S_{\text{final}} + \alpha S_{\text{key}} + \beta S_{\text{cons}}.$$

Here $M$0 measures final answer correctness, $M$1 measures correctness of stored key information in $M$2, and $M$3 measures consistency between stored key information and the final output. The implementation details report $M$4 and $M$5 (Feng et al., 10 Apr 2026).

The rollout reward is then defined piecewise. If the F1 score is not $M$6 and the answer is correct, the reward is $M$7. If the F1 score is $M$8 but the format is correct, the reward is $M$9. If the F1 score is $x$0 and the format is incorrect, the reward is $x$1. This reward is sparse but structured, and it is explicitly intended to encourage correct formatting, correct key saving, and faithful reuse of saved evidence (Feng et al., 10 Apr 2026).

Training all components at once is reported to be unstable. M2R therefore uses curriculum learning in two stages. Stage 1 trains the model to perform macro retrieval, identify answer-aligned evidence, and save it correctly into $x$2. Stage 2 introduces micro retrieval, answer grounding from $x$3, and final answer generation using retrieved key facts. The paper reports that direct joint optimization performs very poorly, and presents this as validation of the curriculum design (Feng et al., 10 Apr 2026).

4. Evaluation protocol and empirical results

The reported experiments train M2R on Qwen2.5-3B-Instruct and Qwen2.5-7B-Instruct, with additional experiments on Llama-3.1-8B-Instruct and Mistral-7B-Instruct. Main training uses the MuSiQue training split with 19,938 samples for 2 epochs. The retriever is E5-base-v2, the knowledge base is a December 2018 Wikipedia snapshot, top-5 passages are retrieved per query, and the system is implemented with FlashRAG (Feng et al., 10 Apr 2026).

Evaluation covers standard multi-hop QA datasets—HotpotQA, 2WikiMultiHopQA, MuSiQue, and Bamboogle—as well as long-form or harder settings including HotpotQA-2Q, HotpotQA-3Q, and extended multi-question versions for all datasets. Baselines include No RAG or Naive Generation, Naive RAG, Iter-RetGen, IRCoT, COFT, SURE, and ReSearch. Reported metrics are Exact Match (EM) and LLM-as-a-Judge (LJ) using GPT-4o-mini with a tailored prompt (Feng et al., 10 Apr 2026).

The main reported numbers for the Qwen models are as follows.

Benchmark	Qwen2.5-3B-Instruct	Qwen2.5-7B-Instruct
HotpotQA	38.70 EM / 56.46 LJ	44.11 / 65.98
2WikiMultiHopQA	40.07 / 48.34	48.89 / 57.01
MuSiQue	20.87 / 32.97	24.12 / 35.44
Bamboogle	39.58 / 47.20	44.56 / 56.89

The paper states that M2R consistently outperforms all baselines, and that the Qwen2.5-7B-Instruct configuration yields the best reported numbers in the table. It also reports that M2R maintains stronger accuracy and lower hallucination than all baselines on concatenated multi-question settings such as HotpotQA-2Q and HotpotQA-3Q, attributing this to repeated re-anchoring of evidence near the answer through micro retrieval (Feng et al., 10 Apr 2026).

The ablation results are central to the interpretation of these gains. A simplified one-shot grounding variant, which feeds all saved key information to the model at once during answering instead of using on-demand micro retrieval, performs worse. A direct optimization baseline that tries to learn macro retrieval, key saving, and micro retrieval jointly is reported to fail badly and to perform worse than even Naive RAG. These findings are used to support two claims internal to the framework: first, that timing of evidence insertion matters, and second, that curriculum learning is essential for stable acquisition of retrieval and grounding skills (Feng et al., 10 Apr 2026).

The reported efficiency overhead is limited. M2R adds only 1–2 extra micro-retrieval calls, corresponding to roughly a 20–30% relative increase in tool invocations, while most cost remains in the macro phase already shared with standard tool-based RAG. End-to-end latency increases by less than 10% on average versus ReSearch, and repository storage is about 50–150 tokens (Feng et al., 10 Apr 2026).

5. Conceptual relation to other micro-macro retrieval paradigms

The provided literature also uses micro-macro decompositions in technically distinct settings, and these cases clarify what is specific to M2R and what is generic to the broader micro-macro pattern.

A physically explicit retrieval-like procedure appears in optical quantum physics. “Observation of micro-macro entanglement of light” constructs an entangled state in which one arm remains at the single-photon level while the other is displaced to about $x$4 photons, and then verifies entanglement by applying the inverse displacement $x$5 and performing full quantum state tomography after the macroscopic state has been returned to the microscopic regime (Lvovsky et al., 2012). In that setting, the retrieval operation is literal de-amplification: information encoded in a macroscopically amplified optical mode is coherently returned to a microscopic state where standard diagnostics can be applied.

A complementary result, “Coarse Graining Makes It Hard to See Micro-Macro Entanglement,” shows that fixed moderate coarse-graining in photon-counting measurements causes the Manhattan distance between the statistics of a unitary entangled cloner and a separable measure-and-prepare cloner to approach zero as system size grows (Raeisi et al., 2011). This suggests that amplification or scaling alone does not guarantee reliable verification: the observability of fine-grained structure depends on how information is read out.

In representation learning, “Hybrid Micro/Macro Level Convolution for Heterogeneous Graph Learning” separates micro-level convolution within the same relation from macro-level convolution across relations, with a weighted residual connection that fuses self-information and aggregated context (Yu et al., 2020). There the terminology refers to two-stage relevance computation rather than answer grounding, but the decomposition is structurally similar: local signals are weighted first, higher-level relation contexts are weighted second, and the final representation depends on both levels.

These uses are not identical to M2R for long-form language generation. Their common pattern is a deliberate separation between fine-grained and coarse-grained operations, followed by a mechanism that reconnects them. In the LLM setting, the reconnecting mechanism is the key-information repository $x$6 and the micro retrieval performed immediately before answer generation (Feng et al., 10 Apr 2026).

6. Misconceptions, limitations, and open directions

Several misconceptions are directly addressed by the reported experiments. M2R is not equivalent to simply increasing the number of retrieval calls, because its reported advantage depends on storing essential results during reasoning and reusing them on demand during answering. It is also not equivalent to feeding the answer phase a larger bundle of saved evidence: the one-shot grounding ablation performs worse than on-demand micro retrieval. Finally, the framework is not presented as something that can be learned stably by naive joint optimization; direct joint optimization performs very poorly, and the curriculum schedule is introduced precisely because evidence extraction and evidence reuse are not acquired reliably at the same time (Feng et al., 10 Apr 2026).

The paper also records several limitations. The rewards are still rule-based rather than learned factuality models. Micro retrieval is implemented as simple dictionary-style lookup, and the authors note that smarter retrieval could improve performance. Proximity control itself could be further optimized dynamically. Extension to richer tools or multimodal settings is explicitly left to future work (Feng et al., 10 Apr 2026).

The broader significance of M2R lies in how it reformulates retrieval augmentation. Instead of treating retrieval as a front-loaded context injection step, it treats retrieval as a staged grounding process: coarse-grained evidence is acquired during reasoning, distilled into answer-aligned entries, and then reintroduced immediately before output. Within the reported experiments, this design is associated with stronger performance on standard multi-hop QA benchmarks, especially in lengthy-context settings, while incurring only modest latency and storage overhead (Feng et al., 10 Apr 2026).