---
title: 'COSMIR: Structured Memory for Long-Context QA'
url: https://www.emergentmind.com/topics/cosmir
type: topic
---

# COSMIR: Structured Memory for Long-Context QA

COSMIR, short for **Chain Orchestrated Structured Memory for Iterative Reasoning over Long Context**, is a **training-free multi-agent framework** for question answering over very long documents. It is designed for settings such as novels and book-length narratives, where a single large language model pass is often unreliable and where common alternatives—retrieval-based context shrinking, enlarged context windows, or staged multi-agent reading—each introduce failure modes of their own. COSMIR preserves chunk-by-chunk, stepwise processing, but replaces free-form inter-agent summaries with a **shared structured memory** and constrains worker behavior to a fixed **Extract \(\rightarrow\) Infer \(\rightarrow\) Refine** micro-cycle, followed by final synthesis by a Manager agent [2510.04568].

## 1. Problem setting and motivating failure modes

Long-context reasoning remains difficult even when a model can technically ingest very long inputs. The core difficulty is not only context length, but also the need to preserve early clues whose relevance may become apparent only much later, reconcile distant evidence iteratively, and avoid dropping details during compression. COSMIR is positioned against three common strategies.

The first strategy is to **shrink the input** through retrieval. In the formulation associated with COSMIR, this risks missing crucial evidence, especially when relevance is indirect or only becomes clear after later passages are read. The second is to **enlarge the context window** and process the document directly; this alleviates truncation, but the framework’s authors argue that selectivity and focus degrade over very large inputs. The third is to **stage multiple agents** over sequential chunks. COSMIR is most directly a response to this third family, especially to **Chain of Agents (CoA)**, where workers pass along free-form summaries.

The critique of staged free-form summarization is central. Two failure modes are emphasized. One is **faulty fact extraction**: a worker may omit material that does not yet appear directly relevant. The other is **fact dropping during sequential processing**: an early worker may preserve a clue, but later workers may compress it away. The paper’s qualitative examples are organized around exactly these failures. In the “Kiara and Carter” example, an early passage describes Kiara meeting a “pale young gentleman,” and only much later is that figure identified as Carter; a system must retain the early event long enough to connect it to the later reveal. In the “Marianne death” example, a staged summary drifts toward thematic context and omits the answer-bearing mechanism of death.

This framing makes COSMIR less a general-purpose agent society than a specific intervention on **propagation-stage information loss**. The architecture is intended to preserve broad, potentially latent evidence while still allowing sequential chunk processing.

## 2. Architecture and formal memory model

COSMIR consists of four components: a **Planner agent**, a **shared structured memory**, **Worker agents**, and a **Manager agent**. Its main formal object is the memory tuple

\[
M := \big\langle \mathcal{Q}, \mathcal{F}_g, \mathcal{F}_i, a \big\rangle,
\]

where \(\mathcal{Q}\) is the set of unresolved sub-questions, \(\mathcal{F}_g\) is the set of gathered facts, \(\mathcal{F}_i\) is the set of inferred facts, and \(a\) is the synthesized answer, which remains empty until the final stage [2510.04568].

The memory is implemented as a **YAML-based external memory** with four top-level fields:

```yaml
questions:
  - "..."
gathered_facts: []
inferred_facts: []
answer: ""
```

This representation is structured in the sense of **slot-structured organization**, not in the sense of a deeply typed database schema. The framework does not define provenance IDs, confidence scores, or a formal contradiction representation. Evidence is stored as textual fact statements in `gathered_facts`; intermediate, grounded conclusions are stored as textual entries in `inferred_facts`; unresolved threads are stored in `questions`; and final synthesis is written to `answer`.

The memory design also includes an explicit budget. The size of \(\mathcal{F}_g\) is constrained to at most a **\(k\)-fraction of the length of a chunk**. If `gathered_facts` exceeds budget, the **oldest facts are pruned** until the memory fits. The pseudocode states this as

\[
\mathcal{F}_g \gets \mathrm{Prune}(\mathcal{F}_g, k).
\]

Only `gathered_facts` is described as being pruned in this manner; the paper does not specify analogous pruning for `inferred_facts`.

This memory design separates extracted evidence from derived claims and from open investigative agenda. That separation underlies the framework’s claims about faithfulness and auditability: the intermediate state is explicit and inspectable rather than latent inside a moving natural-language summary.

## 3. Worker micro-cycle and end-to-end control flow

The operational core of COSMIR is the fixed worker micro-cycle **Extract \(\rightarrow\) Infer \(\rightarrow\) Refine**. A long document is segmented into chunks, and workers process those chunks **left-to-right**. For each chunk \(c_j\), the paper gives the following update pattern:

\[
\Delta \mathcal{F}_g \gets \mathrm{Extract}(c_j,\mathcal{Q})
\]

\[
\mathcal{F}_g \gets \mathcal{F}_g \cup \Delta \mathcal{F}_g
\]

\[
\mathcal{F}_g \gets \mathrm{Prune}(\mathcal{F}_g, k)
\]

\[
\Delta \mathcal{F}_i \gets \mathrm{Infer}(\mathcal{F}_g,\mathcal{F}_i)
\]

\[
\mathcal{F}_i \gets \mathcal{F}_i \cup \Delta \mathcal{F}_i
\]

\[
\mathcal{Q} \gets \mathrm{Refine}(\mathcal{Q},\mathcal{F}_g,\mathcal{F}_i).
\]

The inference basis is explicitly defined as

\[
E=\mathcal{F}_g\cup\mathcal{F}_i.
\]

The **Extract** phase operates over the current chunk, the main query, the current question set, and the current memory. Its prompt requires the worker to extract **ALL relevant facts** from the current chunk and to pay particular attention to named entities, relationships, historical connections, and technical specifications and capabilities. The output is YAML only.

The **Infer** phase consumes \(\mathcal{F}_g\) and \(\mathcal{F}_i\) and produces grounded claims by connecting current evidence and prior inferences. The prompt guidance explicitly mentions connections between entities in different facts, historical relationships, organizational relationships, and timeline connections.

The **Refine** phase updates the open agenda. It removes answered questions and may add new focused questions. This mechanism means COSMIR does not merely carry forward accumulated evidence; it also carries forward an evolving search strategy.

The framework’s end-to-end control flow is therefore:

1. plan the query into sub-questions,
2. initialize memory,
3. process chunks sequentially with Extract, Infer, and Refine,
4. reconstruct the memory tuple after each chunk,
5. synthesize the final answer from the completed memory.

The paper’s Algorithm 1 makes explicit that the shared memory itself is the communication unit between worker invocations.

## 4. Planning, synthesis, and the meaning of “structured memory”

The **Planner** takes the user query \(q\) and converts it into a **small set of concrete, checkable sub-questions**. These are of two kinds. The first are **focused questions**, which directly decompose the user query. The second are **exploratory information nets**, broader questions intended to capture potentially relevant facts that may not appear directly linked at first encounter. The planning prompt instructs the system to identify core entities, central events or relationships, and constraints such as time, location, and chronology, then to build both a **Direct Approach** and a **Decomposed Approach**.

This planning stage is significant because COSMIR does not leave workers to infer the entire search strategy from the main query alone. Instead, the system externalizes a provisional reasoning agenda before chunk processing begins.

The **Manager** receives the final memory \(M\) and computes the answer \(a\), represented in pseudocode as

\[
a \gets \mathrm{SYNTHESIZE}(M).
\]

Its prompt asks it to identify relevant entities in the facts, trace relationships and connections, follow logical chains, and provide a direct concise answer. The output again follows YAML form:

```yaml
answer: "concise answer here"
questions: []
```

The paper’s characterization of structured memory requires some care. A common misconception would be to read the framework as introducing a symbolic database or a formal verification layer. It does not. “Structured” here means that communication is divided into explicit slots—questions, gathered facts, inferred facts, answer—rather than carried by unconstrained prose. There is **no separate validator agent**, **no scoring rule**, **no confidence model**, and **no contradiction-resolution algorithm**. Conflict resolution is implicit in the final synthesis step.

This suggests that COSMIR’s main contribution is procedural and representational discipline rather than formal symbolic reasoning. The framework constrains what each stage is allowed to do and where each type of intermediate content is stored.

## 5. Evaluation on long-context question answering

COSMIR is evaluated on the **long-context QA split of the HELMET benchmark** [2510.04568]. The three datasets are **\(\infty\) bench English QA** (reported as **InfBench-QA**, metric: **ROUGE-F1**), **\(\infty\) bench English MC** (reported as **InfBench-MC**, metric: **Exact Match (EM)**), and **NarrativeQA**, filtered to contexts of at least **256000 tokens** and reported as **NarrativeQA-256k** (metric: **ROUGE-F1**).

The main baselines are **CoA** and **TC (Truncated Context)**, where context is truncated to **128000 tokens** by removing sentences from the middle. All methods are run with **GPT-4.1**, **GPT-4.1-mini**, and **Qwen3-14B**. The reported implementation uses **fixed-length chunking** with **chunk size 64000 tokens**, **maximum summary/memory size 8000 tokens**, and **left-to-right** processing.

The main quantitative results show that COSMIR outperforms both CoA and TC on all reported model-dataset combinations.

| Model | Dataset / metric | TC / CoA / COSMIR |
|---|---|---|
| GPT-4.1 | InfBench-QA / ROUGE-F1 | 36.05 / 47.62 / **50.74** |
| GPT-4.1 | InfBench-MC / EM | 70.31 / 86.03 / **87.33** |
| GPT-4.1 | NarrativeQA-256k / ROUGE-F1 | 28.87 / 35.27 / **37.58** |
| GPT-4.1-mini | InfBench-QA / ROUGE-F1 | 17.59 / 40.47 / **43.56** |
| GPT-4.1-mini | InfBench-MC / EM | 46.28 / 72.49 / **74.23** |
| GPT-4.1-mini | NarrativeQA-256k / ROUGE-F1 | 18.10 / 29.17 / **31.43** |
| Qwen3-14B | InfBench-QA / ROUGE-F1 | 35.99 / 38.12 / **40.76** |
| Qwen3-14B | InfBench-MC / EM | 56.33 / 65.07 / **65.93** |
| Qwen3-14B | NarrativeQA-256k / ROUGE-F1 | 27.37 / 29.53 / **31.14** |

The gains over CoA are largest on the **free-form QA tasks**, especially **InfBench-QA** and **NarrativeQA-256k**. The paper attributes the smaller gains on **InfBench-MC** to the fact that multiple-choice options already guide evidence collection, making CoA less fragile.

The ablation study isolates the **Extract** phase as the key bottleneck. Three configurations are reported: **COSMIR-Qwen3**, **COSMIR-Extract-Qwen3**, and **COSMIR-GPT-4.1**. Weakening only Extract causes substantial regression toward the all-Qwen setting even when Planner, Infer, Refine, and Manager remain strong. This result is important because it shows that structured memory does not eliminate dependence on evidence capture; it reorganizes and stabilizes downstream processing once evidence has been captured.

The paper interprets the consistent gains over CoA as evidence of reduced **propagation-stage information loss**. Strictly speaking, no independent metric with that name is reported. The claim is supported instead by end-task gains and by qualitative case studies.

## 6. Limitations, scope, and interpretation

COSMIR’s limitations are explicit and materially shape how its contributions should be interpreted [2510.04568]. The most important limitation is **dependence on extraction quality**. If Extract misses a crucial fact and that fact does not recur later, subsequent stages generally cannot recover it. The ablation results reinforce this point.

A second limitation is orchestration cost. COSMIR requires **three times as many LLM calls as CoA**, because each chunk is processed through multiple subcalls instead of a single summary generation. This is a substantial systems cost and makes the framework a tradeoff between accuracy, inspectability, and inference budget.

A third limitation concerns memory management. Because the framework prunes the **oldest gathered facts** when the memory exceeds budget, important early evidence can still be lost. The design reduces one form of summary-induced forgetting, but it does not remove memory scarcity.

The evaluation scope is also narrow. The reported experiments cover **long-context QA** only. The paper does not validate the framework on **summarization**, **legal text**, **medical documents**, **code**, or other long-context tasks. It also uses **fixed-size chunks** in original left-to-right order; dynamic chunking and reordering are left as future work.

Methodologically, COSMIR is algorithmic and prompt-driven rather than mathematically optimized. The paper includes the memory tuple and update rules, but there are **no training objectives**, **no optimization losses**, **no confidence scoring rules**, **no contradiction penalties**, and **no formal answer selection rules**. Claims about **faithfulness** are argued from the framework’s structure rather than established by a dedicated faithfulness benchmark. Claims about reduced information loss are likewise interpretive, supported by examples and benchmark improvements rather than by a standalone formal metric.

These limitations clarify what COSMIR is and is not. It is not a formal verification system, not a typed knowledge base, and not a learned memory architecture in the training sense. It is a **training-free long-context reasoning framework** whose novelty lies in replacing free-form summary passing with a shared, inspectable, slot-structured memory and in constraining worker behavior through a fixed reasoning micro-cycle. Within that scope, it offers a specific response to the brittleness of staged long-context processing and a concrete architecture for preserving evidence, inferences, and unresolved questions across iterative reading.

Source: https://www.emergentmind.com/topics/cosmir