COSMIR: Structured Memory for Long-Context QA

• COSMIR is a training-free multi-agent framework for long-context QA that uses a shared, slot-structured memory to retain evidence from lengthy documents.
• It employs a fixed Extract → Infer → Refine cycle, reducing propagation-stage information loss and stabilizing sequential processing.
• Benchmark results show COSMIR outperforms traditional methods, though it requires higher computational cost and is sensitive to extraction quality.

COSMIR, short for Chain Orchestrated Structured Memory for Iterative Reasoning over Long Context, is a training-free multi-agent framework for question answering over very long documents. It is designed for settings such as novels and book-length narratives, where a single LLM pass is often unreliable and where common alternatives—retrieval-based context shrinking, enlarged context windows, or staged multi-agent reading—each introduce failure modes of their own. COSMIR preserves chunk-by-chunk, stepwise processing, but replaces free-form inter-agent summaries with a shared structured memory and constrains worker behavior to a fixed Extract $\rightarrow$ Infer $\rightarrow$ Refine micro-cycle, followed by final synthesis by a Manager agent (Gupta et al., 6 Oct 2025).

1. Problem setting and motivating failure modes

Long-context reasoning remains difficult even when a model can technically ingest very long inputs. The core difficulty is not only context length, but also the need to preserve early clues whose relevance may become apparent only much later, reconcile distant evidence iteratively, and avoid dropping details during compression. COSMIR is positioned against three common strategies.

The first strategy is to shrink the input through retrieval. In the formulation associated with COSMIR, this risks missing crucial evidence, especially when relevance is indirect or only becomes clear after later passages are read. The second is to enlarge the context window and process the document directly; this alleviates truncation, but the framework’s authors argue that selectivity and focus degrade over very large inputs. The third is to stage multiple agents over sequential chunks. COSMIR is most directly a response to this third family, especially to Chain of Agents (CoA), where workers pass along free-form summaries.

The critique of staged free-form summarization is central. Two failure modes are emphasized. One is faulty fact extraction: a worker may omit material that does not yet appear directly relevant. The other is fact dropping during sequential processing: an early worker may preserve a clue, but later workers may compress it away. The paper’s qualitative examples are organized around exactly these failures. In the “Kiara and Carter” example, an early passage describes Kiara meeting a “pale young gentleman,” and only much later is that figure identified as Carter; a system must retain the early event long enough to connect it to the later reveal. In the “Marianne death” example, a staged summary drifts toward thematic context and omits the answer-bearing mechanism of death.

This framing makes COSMIR less a general-purpose agent society than a specific intervention on propagation-stage information loss. The architecture is intended to preserve broad, potentially latent evidence while still allowing sequential chunk processing.

2. Architecture and formal memory model

COSMIR consists of four components: a Planner agent, a shared structured memory, Worker agents, and a Manager agent. Its main formal object is the memory tuple

$$M := \big\langle \mathcal{Q}, \mathcal{F}_g, \mathcal{F}_i, a \big\rangle,$$

where $\mathcal{Q}$ is the set of unresolved sub-questions, $\mathcal{F}_g$ is the set of gathered facts, $\mathcal{F}_i$ is the set of inferred facts, and $a$ is the synthesized answer, which remains empty until the final stage (Gupta et al., 6 Oct 2025).

The memory is implemented as a YAML-based external memory with four top-level fields:

$$M := \big\langle \mathcal{Q}, \mathcal{F}_g, \mathcal{F}_i, a \big\rangle,$$8

This representation is structured in the sense of slot-structured organization, not in the sense of a deeply typed database schema. The framework does not define provenance IDs, confidence scores, or a formal contradiction representation. Evidence is stored as textual fact statements in gathered_facts; intermediate, grounded conclusions are stored as textual entries in inferred_facts; unresolved threads are stored in questions; and final synthesis is written to answer.

The memory design also includes an explicit budget. The size of $\mathcal{F}_g$ is constrained to at most a $k$-fraction of the length of a chunk. If gathered_facts exceeds budget, the oldest facts are pruned until the memory fits. The pseudocode states this as

$$\mathcal{F}_g \gets \mathrm{Prune}(\mathcal{F}_g, k).$$

Only gathered_facts is described as being pruned in this manner; the paper does not specify analogous pruning for inferred_facts.

This memory design separates extracted evidence from derived claims and from open investigative agenda. That separation underlies the framework’s claims about faithfulness and auditability: the intermediate state is explicit and inspectable rather than latent inside a moving natural-language summary.

3. Worker micro-cycle and end-to-end control flow

The operational core of COSMIR is the fixed worker micro-cycle Extract $\rightarrow$0 Infer $\rightarrow$1 Refine. A long document is segmented into chunks, and workers process those chunks left-to-right. For each chunk $\rightarrow$2, the paper gives the following update pattern:

$\rightarrow$3

$\rightarrow$4

$\rightarrow$5

$\rightarrow$6

$\rightarrow$7

$\rightarrow$8

The inference basis is explicitly defined as

$\rightarrow$9

The Extract phase operates over the current chunk, the main query, the current question set, and the current memory. Its prompt requires the worker to extract ALL relevant facts from the current chunk and to pay particular attention to named entities, relationships, historical connections, and technical specifications and capabilities. The output is YAML only.

The Infer phase consumes $$M := \big\langle \mathcal{Q}, \mathcal{F}_g, \mathcal{F}_i, a \big\rangle,$$0 and $$M := \big\langle \mathcal{Q}, \mathcal{F}_g, \mathcal{F}_i, a \big\rangle,$$1 and produces grounded claims by connecting current evidence and prior inferences. The prompt guidance explicitly mentions connections between entities in different facts, historical relationships, organizational relationships, and timeline connections.

The Refine phase updates the open agenda. It removes answered questions and may add new focused questions. This mechanism means COSMIR does not merely carry forward accumulated evidence; it also carries forward an evolving search strategy.

The framework’s end-to-end control flow is therefore:

1. plan the query into sub-questions,
2. initialize memory,
3. process chunks sequentially with Extract, Infer, and Refine,
4. reconstruct the memory tuple after each chunk,
5. synthesize the final answer from the completed memory.

The paper’s Algorithm 1 makes explicit that the shared memory itself is the communication unit between worker invocations.

4. Planning, synthesis, and the meaning of “structured memory”

The Planner takes the user query $$M := \big\langle \mathcal{Q}, \mathcal{F}_g, \mathcal{F}_i, a \big\rangle,$$2 and converts it into a small set of concrete, checkable sub-questions. These are of two kinds. The first are focused questions, which directly decompose the user query. The second are exploratory information nets, broader questions intended to capture potentially relevant facts that may not appear directly linked at first encounter. The planning prompt instructs the system to identify core entities, central events or relationships, and constraints such as time, location, and chronology, then to build both a Direct Approach and a Decomposed Approach.

This planning stage is significant because COSMIR does not leave workers to infer the entire search strategy from the main query alone. Instead, the system externalizes a provisional reasoning agenda before chunk processing begins.

The Manager receives the final memory $$M := \big\langle \mathcal{Q}, \mathcal{F}_g, \mathcal{F}_i, a \big\rangle,$$3 and computes the answer $$M := \big\langle \mathcal{Q}, \mathcal{F}_g, \mathcal{F}_i, a \big\rangle,$$4, represented in pseudocode as

$$M := \big\langle \mathcal{Q}, \mathcal{F}_g, \mathcal{F}_i, a \big\rangle,$$5

Its prompt asks it to identify relevant entities in the facts, trace relationships and connections, follow logical chains, and provide a direct concise answer. The output again follows YAML form:

$$M := \big\langle \mathcal{Q}, \mathcal{F}_g, \mathcal{F}_i, a \big\rangle,$$9

The paper’s characterization of structured memory requires some care. A common misconception would be to read the framework as introducing a symbolic database or a formal verification layer. It does not. “Structured” here means that communication is divided into explicit slots—questions, gathered facts, inferred facts, answer—rather than carried by unconstrained prose. There is no separate validator agent, no scoring rule, no confidence model, and no contradiction-resolution algorithm. Conflict resolution is implicit in the final synthesis step.

This suggests that COSMIR’s main contribution is procedural and representational discipline rather than formal symbolic reasoning. The framework constrains what each stage is allowed to do and where each type of intermediate content is stored.

5. Evaluation on long-context question answering

COSMIR is evaluated on the long-context QA split of the HELMET benchmark (Gupta et al., 6 Oct 2025). The three datasets are $$M := \big\langle \mathcal{Q}, \mathcal{F}_g, \mathcal{F}_i, a \big\rangle,$$6 bench English QA (reported as InfBench-QA, metric: ROUGE-F1), $$M := \big\langle \mathcal{Q}, \mathcal{F}_g, \mathcal{F}_i, a \big\rangle,$$7 bench English MC (reported as InfBench-MC, metric: Exact Match (EM)), and NarrativeQA, filtered to contexts of at least 256000 tokens and reported as NarrativeQA-256k (metric: ROUGE-F1).

The main baselines are CoA and TC (Truncated Context), where context is truncated to 128000 tokens by removing sentences from the middle. All methods are run with GPT-4.1, GPT-4.1-mini, and Qwen3-14B. The reported implementation uses fixed-length chunking with chunk size 64000 tokens, maximum summary/memory size 8000 tokens, and left-to-right processing.

The main quantitative results show that COSMIR outperforms both CoA and TC on all reported model-dataset combinations.

Model	Dataset / metric	TC / CoA / COSMIR
GPT-4.1	InfBench-QA / ROUGE-F1	36.05 / 47.62 / 50.74
GPT-4.1	InfBench-MC / EM	70.31 / 86.03 / 87.33
GPT-4.1	NarrativeQA-256k / ROUGE-F1	28.87 / 35.27 / 37.58
GPT-4.1-mini	InfBench-QA / ROUGE-F1	17.59 / 40.47 / 43.56
GPT-4.1-mini	InfBench-MC / EM	46.28 / 72.49 / 74.23
GPT-4.1-mini	NarrativeQA-256k / ROUGE-F1	18.10 / 29.17 / 31.43
Qwen3-14B	InfBench-QA / ROUGE-F1	35.99 / 38.12 / 40.76
Qwen3-14B	InfBench-MC / EM	56.33 / 65.07 / 65.93
Qwen3-14B	NarrativeQA-256k / ROUGE-F1	27.37 / 29.53 / 31.14

The gains over CoA are largest on the free-form QA tasks, especially InfBench-QA and NarrativeQA-256k. The paper attributes the smaller gains on InfBench-MC to the fact that multiple-choice options already guide evidence collection, making CoA less fragile.

The ablation study isolates the Extract phase as the key bottleneck. Three configurations are reported: COSMIR-Qwen3, COSMIR-Extract-Qwen3, and COSMIR-GPT-4.1. Weakening only Extract causes substantial regression toward the all-Qwen setting even when Planner, Infer, Refine, and Manager remain strong. This result is important because it shows that structured memory does not eliminate dependence on evidence capture; it reorganizes and stabilizes downstream processing once evidence has been captured.

The paper interprets the consistent gains over CoA as evidence of reduced propagation-stage information loss. Strictly speaking, no independent metric with that name is reported. The claim is supported instead by end-task gains and by qualitative case studies.

6. Limitations, scope, and interpretation

COSMIR’s limitations are explicit and materially shape how its contributions should be interpreted (Gupta et al., 6 Oct 2025). The most important limitation is dependence on extraction quality. If Extract misses a crucial fact and that fact does not recur later, subsequent stages generally cannot recover it. The ablation results reinforce this point.

A second limitation is orchestration cost. COSMIR requires three times as many LLM calls as CoA, because each chunk is processed through multiple subcalls instead of a single summary generation. This is a substantial systems cost and makes the framework a tradeoff between accuracy, inspectability, and inference budget.

A third limitation concerns memory management. Because the framework prunes the oldest gathered facts when the memory exceeds budget, important early evidence can still be lost. The design reduces one form of summary-induced forgetting, but it does not remove memory scarcity.

The evaluation scope is also narrow. The reported experiments cover long-context QA only. The paper does not validate the framework on summarization, legal text, medical documents, code, or other long-context tasks. It also uses fixed-size chunks in original left-to-right order; dynamic chunking and reordering are left as future work.

Methodologically, COSMIR is algorithmic and prompt-driven rather than mathematically optimized. The paper includes the memory tuple and update rules, but there are no training objectives, no optimization losses, no confidence scoring rules, no contradiction penalties, and no formal answer selection rules. Claims about faithfulness are argued from the framework’s structure rather than established by a dedicated faithfulness benchmark. Claims about reduced information loss are likewise interpretive, supported by examples and benchmark improvements rather than by a standalone formal metric.

These limitations clarify what COSMIR is and is not. It is not a formal verification system, not a typed knowledge base, and not a learned memory architecture in the training sense. It is a training-free long-context reasoning framework whose novelty lies in replacing free-form summary passing with a shared, inspectable, slot-structured memory and in constraining worker behavior through a fixed reasoning micro-cycle. Within that scope, it offers a specific response to the brittleness of staged long-context processing and a concrete architecture for preserving evidence, inferences, and unresolved questions across iterative reading.