---
title: 'MQMTAR: Multi-Query Multi-Token Recall'
url: https://www.emergentmind.com/topics/multi-query-multi-token-associative-recall-mqmtar
type: topic
---

# MQMTAR: Multi-Query Multi-Token Recall

Multi-query Multi-token Associative Recall (MQMTAR) denotes a retrieval regime in which a model must preserve many key–value bindings across a sequence, survive distractor content and varying interaction distances, and answer multiple later queries from the same prefix. In the cited literature, the formal synthetic benchmark is **Multi-Query Associative Recall (MQAR)**, introduced to correct the mismatch between earlier single-query associative-recall tests and the demands of natural language; the “multi-token” qualifier refers chiefly to the motivating linguistic phenomena—repeated bigrams, repeated phrases, names, and other multi-token expressions—rather than to a separate span-retrieval formalism [2312.04927, 2605.06997].

## 1. Terminological scope and historical placement

Earlier associative-recall benchmarks typically presented **one query at a fixed position**, used a **small vocabulary**, and asked the model to retrieve a single associated value. MQAR was introduced because these conditions did not match the retrieval structure of real language, where a model may need to perform **many recall operations in one forward pass**, at **different positions**, over a **large vocabulary**, and sometimes in connection with repeated phrases or other multi-token material rather than isolated symbols [2312.04927].

Within this literature, MQAR is the formal task name. The later Echo work treats **MQMTAR / Multi-Query Associative Recall** as the failure mode exposed when a sequence contains many KV pairs, then a distractor gap, then several queries asking for associated values. In that usage, MQMTAR functions less as a distinct benchmark definition than as a descriptive emphasis on the same multi-query retrieval problem under long-context conditions [2605.06997].

This distinction matters conceptually. The benchmark formalism is token-level, but the motivating linguistic examples are often multi-token. As a result, the phrase MQMTAR is best understood as referring to multi-query associative recall in settings where the underlying language phenomena may involve multi-token spans, even though the core synthetic objective remains retrieval of tokenized key–value associations.

## 2. Formal task structure

MQAR is formulated over an input sequence of interleaved keys, values, and queries:
\[
\bm{u}[0 \cdots N-1] \triangleq \left\{\paren{\bm{k}_0, \bm{v}_0, \bm{q}_0}, \ldots, \paren{\bm{k}_{\frac{N}{3}-1}, \bm{v}_{\frac{N}{3}-1}, \bm{q}_{\frac{N}{3}-1}}\right\}.
\]
Each element is drawn from a vocabulary \(C\), and the goal is, for each query position, to determine whether there exists an earlier key matching that query and, if so, to output the corresponding value:
\[
\text{for each } 1 \le i \le \frac{N}{3}-1,\ \text{check whether there exists } 0 \le j < i \text{ such that } \bm{q}_i \equiv \bm{k}_j,\ \text{and if so, output } \bm{v}_j.
\]

The role positions are fixed modulo 3:
\[
\begin{aligned}
\calK &= \{i \in \{0, \ldots, N-1\} \lvert\ i \equiv 0 \mod{3}\},\\
\calV &= \{i \in \{0, \ldots, N-1\} \lvert\ i \equiv 1 \mod{3}\},\\
\calQ &= \{i \in \{0, \ldots, N-1\} \lvert\ i \equiv 2 \mod{3}\}.
\end{aligned}
\]
These are encoded as one-hot blocks:
\[
\bm{u}[i,:] \equiv
\begin{cases}
[\bm{k}_i:\bm{0}^c:\bm{0}^c] &\text{ if } i \in \calK \\
[\bm{0}^c:\bm{v}_i:\bm{0}^c] &\text{ if } i \in \calV \\
[\bm{0}^c:\bm{0}^c:\bm{q}_i] &\text{ if } i \in \calQ
\end{cases}
\]
The crucial shift from classical associative recall is that retrieval is required **for each query position**, not for a single designated query.

The Echo paper instantiates this family of tasks in a fine-tuning grid with the number of KV pairs \(M \in \{4, 8, 16, 32\}\) and distractor gap length from 64 to 4,096 tokens. Each configuration is trained and evaluated independently, so the benchmark directly tests whether a model can retrieve bindings across long gaps and multiple stored pairs rather than generalizing from a different setting [2605.06997].

## 3. Why the benchmark was introduced

The empirical motivation for MQAR is a mismatch between what older synthetic tests measure and what language models appear to need in actual text. In Zoology, attention-free gated-convolution architectures could match attention on standard synthetic associative-recall tests, yet still underperformed on language modeling. Fine-grained analysis showed that **82% of the perplexity gap** between attention and gated convolutions on the Pile was explained by **AR Hits**, defined as tokens that complete a repeated bigram seen in context but not heavily memorized during training [2312.04927].

The motivating language phenomena are notably richer than the synthetic benchmark itself. The paper uses examples such as “Hakuna Matata” / “no worries,” names like “Tim Rice,” and repeated phrases and technical terms in ArXiv, GitHub, and StackExchange. It also reports that repeated bigrams in real corpora often occur at different distances, with a **power-law distribution of gaps**. These observations explain why a benchmark with one query, one lookup, and a tiny symbol table can overstate the recall competence of efficient architectures.

MQAR therefore isolates four demands that earlier tests largely omitted: **multiple recalls in one pass**, **different key-query distances**, **the need to keep many associations accessible simultaneously**, and **large-vocabulary token identities**. The “multi-token” reading of MQMTAR enters here. MQAR is formally token-level, but it was motivated by natural-language recall patterns that frequently involve multi-token expressions. This suggests that MQAR is a proxy for a broader family of retrieval phenomena rather than an exhaustive description of them.

The same literature also marks a limitation. The repeated-bigram heuristic does not capture fuzzier or more abstract recall, such as synonyms or higher-level concepts. MQMTAR, in its current usage, is therefore narrower than “long-range memory” in the most general linguistic sense.

## 4. Empirical behavior across attention, gated convolutions, and sparse hybrids

On the Pile, after pretraining on 10B tokens, attention outperformed the strongest gated-convolution baselines by up to **2.1 perplexity points**. At the 360M scale, the gap was especially concentrated on AR Hits, and attention maintained a marked advantage over Hyena, RWKV, H3, and Long Conv [2312.04927].

| Model | Pile PPL at 360M | AR Hits log-PPL at 360M |
|---|---:|---:|
| Attention | 9.44 | 1.98 |
| Hyena | 10.07 | 3.83 |
| RWKV | 9.79 | 3.82 |
| H3 | 10.38 | 4.81 |
| Long Conv | 13.13 | — |

At larger scale, the gap persisted. At **1.4B** parameters, attention achieved **8.19 PPL** and Hyena **9.65 PPL** on the Pile; on AR Hits specifically, the comparison was **1.91** versus **3.43**. The paper also reports that a **70M attention model** outperformed a **1.4B Hyena model** on AR slice perplexity, and characterizes the smaller attention model as “a full perplexity point better” on AR. In sequence-length scaling, attention solved MQAR with model dimension essentially **independent of sequence length**, whereas gated convolutions required model dimension that grows at least **linearly with sequence length** to solve MQAR reliably. As the number of required recalls per example increased, RWKV degraded sharply while attention remained stable [2312.04927].

Zoology also evaluated simple convolution-attention hybrids. The best results came from **input-dependent sparse attention**, not from random sparsity. At 360M scale, the reported overall PPL values were **11.01** for pure BaseConv, **12.94** for random selection, **9.54** for programmatic selection, **9.59** for learned selection, and **8.59** for a full attention hybrid. On AR Hits, programmatic selection closed **85% of the gap** between pure BaseConv and attention, while learned selection closed **72%** using only \(k=256\) attention positions per example. The abstract’s summary is that such hybrids can close **97.4% of the gap to attention** while maintaining **sub-quadratic scaling** [2312.04927].

These results establish the central empirical lesson of the benchmark family: retrieval quality on multi-query recall, rather than generic sequence mixing alone, is a principal bottleneck for efficient attention-free language models.

## 5. Constant-memory retrieval and the Echo architecture

Echo addresses the same retrieval problem from a different direction. It is a KV-cache-free associative recall architecture built around **Spectral Koopman Attention (SKA)**, used as a drop-in replacement for attention layers inside an otherwise Mamba-2-based model. The key claim is that content-based retrieval can be implemented from streaming sufficient statistics rather than from a KV cache that grows as \(O(Td)\) with sequence length \(T\) [2605.06997].

For each token, SKA projects keys, queries, and values as
\[
z_t = W_k x_t \in \mathbb{R}^r,\quad z_t^q = W_q x_t \in \mathbb{R}^r,\quad v_t = W_v x_t \in \mathbb{R}^P.
\]
It then applies sequence-max normalization,
\[
\hat{z}_t = z_t / m,\qquad \hat{z}_t^q = z_t^q / m,\qquad m = \max_{s\le T}\|z_s\|_2,
\]
and accumulates additive statistics
\[
G = \sum_t \hat{z}_t \hat{z}_t^\top + \varepsilon I_r,\qquad
M = \sum_t \hat{z}_t \hat{z}_{t-1}^\top,\qquad
C_v = \sum_t v_t \hat{z}_t^\top.
\]
The value map is obtained by ridge regression,
\[
B_v = \arg\min_B \sum_t \|v_t - B\hat{z}_t\|^2 + \varepsilon\|B\|_F^2,
\qquad
B_v = C_v \tilde{G}^{-1},\ \tilde{G}=G+\varepsilon I_r.
\]
With Cholesky factorization \(\tilde{G}=LL^\top\), the whitened Koopman operator is
\[
A_w = L^{-1} M L^{-\top},
\]
and the query is whitened, power-filtered, and unwhitened:
\[
w_q = L^{-1}\hat{z}^q,\qquad w_f = A_w^K w_q,\qquad z_f = L w_f.
\]
Retrieval is then
\[
\hat{y} = \eta\, B_v z_f,
\]
or equivalently
\[
\hat{y}_{\mathrm{SKA}(q)=\eta\, C_v \tilde{G}^{-1}\Phi_K q,\qquad \Phi_K = L\hat{A}_w^K L^{-1}.
\]
When \(K=0\), SKA reduces to exact ridge retrieval.

The memory state per head is fixed:
\[
G \in \mathbb{R}^{r\times r},\quad M \in \mathbb{R}^{r\times r},\quad C_v \in \mathbb{R}^{P\times r},
\]
plus the previous key and a scalar norm, for a total of
\[
2r^2 + Pr + r + 1
\]
floats per head, independent of \(T\). The paper states that the 50M model’s total SKA state is about **77 KB in FP32** [2605.06997].

On the 50M MQAR benchmark, the contrast is sharp. A pure Mamba-2 model remained near chance, around **\(\sim 3\%\)**, across every configuration. Mamba-2 + Attention reached **100% on all but one cell**, degrading at the largest gap with \(M=4\). Mamba-2 + SKA achieved **100% on every cell tested**, including distractor gaps of **4,096** tokens with **32** KV pairs. In smaller synthetic transfer experiments, average accuracy was **54.4** for SSM, **76.2** for SSM+Attn, and **81.1** for SSM+SKA; the SSM+SKA model was reported as best on MQAR (**82.2**), System Prompt (**85.3**), ToolTrace (**80.5**), NIAH (**84.2**), MultiHop (**76.7**), and CommonWord (**74.1**). In length generalization for needle-in-a-haystack trained at length 64, SSM dropped to **2.0%** at 2048 and 4096, SSM+Attn to **42.5%** at 2048 and **5.0%** at 4096, while SSM+SKA remained at **81.2%** at 2048 and **65.1%** at 4096 [2605.06997].

Ablations attribute the retrieval gain primarily to the **spectral operator / SKA mechanism**, not to the action mask. Additional reported choices include independent projections for SKA, preference for sequence-max normalization over per-token \(\ell_2\) normalization, **power filter order \(K=2\)** as a sweet spot, ungated additive injection over sigmoid gating, and spectral normalization with learned \(\gamma\) to stabilize the operator [2605.06997].

## 6. Interpretation, misconceptions, and research significance

A common misconception is that strong performance on classical synthetic associative recall implies strong long-context recall in language models. The Zoology results reject that equivalence. Gated convolutions could solve older synthetic tests but still exhibited a substantial recall deficit on language modeling, with **82% of the average quality gap** explained by AR Hits. The benchmark was designed precisely to expose that mismatch [2312.04927].

A second misconception is that MQMTAR is already a general multi-token span-retrieval benchmark. The cited literature does not support that reading. MQAR is explicitly a **token-level** formal task. Multi-tokenity enters through the language phenomena used to motivate the task—repeated phrases, entities, and bigrams—and through the observation that real text often requires recall over such structures. This suggests that MQMTAR names an empirical problem class broader than the exact synthetic objective, not a completed formalism for arbitrary span retrieval.

A third misconception is that constant-memory recurrence and content-addressed retrieval are interchangeable. Echo’s results argue otherwise. Pure Mamba-2 maintains \(O(1)\) memory per step but exhibits a structural **memory cliff**, whereas attention solves retrieval by retaining a KV cache, and SKA attempts to recover content-based lookup from constant-size sufficient statistics. In this framing, the core issue is not merely model size or recurrence depth but the mechanism by which token interactions are represented and recovered [2605.06997].

The broader significance of MQMTAR lies in its role as a diagnostic and design target. In Zoology, it identifies recall as the bottleneck separating efficient convolutional language models from attention-based ones. In Echo, it motivates a cache-free retrieval mechanism intended for long chain-of-thought reasoning, agentic tool-calling, and related long-context settings. A plausible implication is that future work on “multi-token” associative recall will need to move beyond token-level proxies toward explicit span-level or compositional retrieval objectives while preserving the benchmark’s central pressure points: many bindings, long distractor gaps, variable distances, and multiple queries from the same prefix.

Source: https://www.emergentmind.com/topics/multi-query-multi-token-associative-recall-mqmtar