Reactive Transformer (RxT) Architecture

• Reactive Transformer (RxT) is a stateful, event-driven dialogue model that replaces full transcript reprocessing with a fixed-size Short-Term Memory, reducing computational cost from O(N²·T) to O(N·T).
• The architecture decouples response generation from memory consolidation using a generator-decoder, Memory Encoder, and Memory Attention network to asynchronously update the persistent state.
• RxT demonstrates improved efficiency and conversational coherence with lower perplexity and stable inference latency, making long-dialogue interactions economically viable.

Searching arXiv for the cited RxT paper and closely related reactive transformer-based work. Reactive Transformer (RxT) is a stateful, event-driven alternative to standard conversational Transformers, introduced to address the claim that ordinary chatbot-style Transformer deployments are fundamentally stateless across turns and therefore must repeatedly reprocess the full dialogue history at every user interaction (Filipek, 3 Oct 2025). In this formulation, each user message is treated as a discrete event rather than as an appended segment of an ever-growing transcript. RxT replaces brute-force transcript replay with a fixed-size Short-Term Memory (STM), decouples response generation from memory consolidation, and is presented as shifting total user-facing conversation cost from $O(N^2 \cdot T)$ to $O(N \cdot T)$ for $N$ turns of average length $T$ (Filipek, 3 Oct 2025).

1. Conceptual reorientation from stateless sequence processing to event-driven dialogue

The central premise of RxT is that standard conversational Transformer usage is architecturally mismatched to real-time dialogue. In ordinary decoder-only deployments, the model is given the entire dialogue history again at every turn so that it can recover prior context from scratch. The stated consequence is that both inference latency and token cost rise with accumulated history, and because attention over a sequence is quadratic in sequence length $L$, the total user-facing cost across a conversation scales as $O(N^2 \cdot T)$ rather than linearly in the number of interactions (Filipek, 3 Oct 2025).

RxT reframes dialogue as a continuous interaction loop. Instead of modeling a conversation as a single static sequence, it treats each conversational turn as an event. At turn $t$, the model processes only the current user query $X_t$, conditions on the previous memory state $STM_{t-1}$, generates the response $Y_t$, and then performs memory consolidation asynchronously. The paper presents this as a shift from a data-driven paradigm to an event-driven paradigm, with the explicit goal of making latency effectively constant with respect to dialogue depth and making long-form conversations economically viable (Filipek, 3 Oct 2025).

This design implies a different notion of conversational state. Context is not reconstructed from a text buffer but maintained as a persistent internal state. The paper argues that this is not merely an efficiency improvement but a conceptual correction for real-time dialogue, because the model no longer re-derives context from scratch at each step (Filipek, 3 Oct 2025).

2. Core architecture and information flow

Architecturally, RxT is an encoder-decoder system reorganized around dialogue turns. Its three principal components are the generator-decoder, the Memory Encoder, and the Memory Attention network. The generator-decoder performs autoregressive response generation; the Memory Encoder compresses the completed interaction; and the Memory Attention network writes new information into the STM after the response has already been emitted (Filipek, 3 Oct 2025).

At turn $O(N \cdot T)$0, the generator-decoder receives the current query $O(N \cdot T)$1 and conditions on $O(N \cdot T)$2 through a Memory Cross-Attention mechanism. In each decoder layer, the sequence is described as masked self-attention over the current interaction, memory cross-attention to retrieve relevant context from STM, and then a feed-forward network. The decoder therefore treats the current query and the response under construction as the active interaction, while STM supplies persistent context from earlier turns. The paper also states that the decoder’s feed-forward networks are implemented with Mixture-of-Experts to expand capacity without exploding the parameter count (Filipek, 3 Oct 2025).

The memory read operation is formalized as cross-attention in which decoder hidden states query the previous STM: $O(N \cdot T)$3 A specific design choice is that positional encodings are applied to decoder queries but not to memory keys and values, because the STM is not intended to function as an ordered token sequence. It is meant to store content rather than sequence position (Filipek, 3 Oct 2025).

After generation, the Memory Encoder processes the concatenation $O(N \cdot T)$4 and produces encoded data $O(N \cdot T)$5. The Memory Attention network then updates the STM using the inverse attention pattern: $O(N \cdot T)$6 In this formulation, memory slots actively pull the information they need from the newly encoded interaction. The paper describes multiple variants of the memory-update block: a simple version, a memory self-attention version, an interlayer memory attention version, and gated versions that regulate retention and overwrite (Filipek, 3 Oct 2025).

3. Short-Term Memory as bounded persistent state

The STM is the persistent state of the model, but it is not a transcript cache or a growing token history. It is described as a fixed-size bank of learnable vectors organized into layers aligned with the model’s depth. This bounded design is essential to the claim that memory access cost does not grow with the number of prior conversational turns (Filipek, 3 Oct 2025).

Because the STM is fixed-size and structured, RxT explicitly accepts memory compression rather than full token preservation. The paper treats this as a deliberate design choice: the architecture is not trying to retain all tokens forever, but instead to preserve relevant state in a bounded representation. That boundedness is presented as a strength for efficiency and latency, although it also creates an information bottleneck relative to unlimited transcript replay (Filipek, 3 Oct 2025).

A particularly important stabilization mechanism is the residual gate for memory updates. Rather than replacing or naively adding the update, RxT interpolates between the previous state and the new information: $O(N \cdot T)$7 where $O(N \cdot T)$8 is a learned gate, typically produced with a sigmoid. The paper also gives a tanh-based alternative: $O(N \cdot T)$9 The stated purpose of gating is to avoid catastrophic forgetting and to prevent memory magnitude from drifting or exploding across many turns. In the paper’s terminology, RxT thereby manages plasticity explicitly: old context is preserved unless the latest interaction is sufficiently important to warrant change (Filipek, 3 Oct 2025).

4. Operational cycle, latency model, and computational scaling

A single user turn in RxT follows a specific event-driven cycle. First, the user submits $N$0. The decoder performs a prompt pass and then autoregressively generates $N$1, attending both to its self-attention cache for the active interaction and to a precomputed memory cross-attention cache derived from $N$2. Because $N$3 remains fixed during generation, its key/value projections can be pre-cached before token generation begins, which the paper identifies as an implementation optimization that avoids redundant computation for each generated token (Filipek, 3 Oct 2025).

Only after the answer has been completed and streamed to the user does the system process $N$4 through the Memory Encoder and Memory Attention network to produce $N$5. That updated STM then becomes available for the next turn. The decisive architectural claim is that memory consolidation does not block response generation, so perceived latency is decoupled from memory-update complexity (Filipek, 3 Oct 2025).

This decoupling is the basis of RxT’s scaling argument. In standard decoder-only dialogue, the prompt phase at turn $N$6 must reprocess an ever-growing sequence of length roughly $N$7, so both prompt cost and generation cost worsen with dialogue depth. RxT instead processes only the current query plus a fixed-size memory state, so the per-turn cost is treated as effectively constant with respect to the number of previous turns. The paper summarizes the resulting shift as

$N$8

It further states that the asynchronous memory update cost depends on the memory size $N$9 rather than conversation length, and that this cost is off the critical path for user-facing latency (Filipek, 3 Oct 2025).

The latency benchmark reported in the paper is framed as “constant-time inference latency” in the practical sense that prompt-phase latency does not rise as dialogue depth increases. In the reported benchmark, the stateless baseline’s prompt latency rose from about 0.09 s at step 1 to over 0.22 s by step 8, while RxT stayed near 0.06 s across all steps. The paper explicitly notes that this is not constant time in the strict theoretical sense that every operation is $T$0, but rather stable response-start latency as the conversation grows (Filipek, 3 Oct 2025).

5. Training protocol and proof-of-concept evaluation

The paper states that RxT cannot be trained well end-to-end in a naïve way because its asynchronous memory system introduces cold-start failure modes: the decoder may learn to ignore noisy memory, or the memory attention pathway may converge to uninformative states. To address this, it proposes a four-stage supervised pipeline: joint language pretraining, joint interaction fine-tuning, self-supervised memory-attention pretraining, and supervised memory-aware training over multi-step dialogues (Filipek, 3 Oct 2025).

The proof-of-concept experiments are synthetic but structured around memory-aware dialogue. RxT variants of different sizes were trained and compared against a decoder-only Transformer baseline of comparable scale. The baseline was 22M parameters, while the RxT variants were 12M, 26M, 100M, and 160M. Training used TinyStories as a general language corpus and then custom multi-turn datasets derived from it. All models used the paper’s Sparse Query Attention implementation, and the decoder side used Mixture-of-Experts layers (Filipek, 3 Oct 2025).

Evaluation measured perplexity, next-token accuracy, and a custom “MRL Reward Score” intended to capture conversational quality and coherence across turns. On the multi-step dialogue test set, the 12M RxT model achieved perplexity 2.74 versus 4.37 for the 22M stateless baseline, and the larger RxT models improved further to 2.18 for the 160M version. Accuracy for RxT was reported around 80–82% versus 55% for the baseline. On the conversational coherence metric, the baseline scored a mean reward of 2.4, while RxT variants ranged from 3.1 to 3.8, with better minimum scores as well. The latency benchmark is used to reinforce the architectural claim that fixed-size memory and asynchronous state update stabilize prompt latency over longer dialogues (Filipek, 3 Oct 2025).

These experiments are explicitly characterized as proof-of-concept evidence. The paper does not position them as a definitive large-scale benchmark, but rather as an initial validation that a stateful, event-driven conversational architecture can outperform a stateless baseline in both quality and efficiency under synthetic multi-turn conditions (Filipek, 3 Oct 2025).

6. Scope, tradeoffs, and relation to other reactive transformer-based work

RxT is explicitly designed for conversational and interaction-centric tasks rather than one-off single-step generation, and it is not positioned as a drop-in replacement for general-purpose completion models (Filipek, 3 Oct 2025). This scope restriction is significant because the architecture’s advantages derive from persistent state, asynchronous consolidation, and repeated interaction cycles. A plausible implication is that its benefits are strongest in settings where response generation and state maintenance recur over long horizons.

The principal tradeoff is the fixed-size STM. The paper presents bounded state as a feature because it preserves relevant context instead of replaying raw history, but this also means RxT compresses dialogue and therefore cannot retain all tokens indefinitely (Filipek, 3 Oct 2025). A second tradeoff is implementation complexity. The custom training curriculum, the separate Memory Encoder and Memory Attention pathways, and the asynchronous update schedule all complicate optimization relative to a standard stateless Transformer (Filipek, 3 Oct 2025).

The broader term “reactive” also appears in adjacent Transformer-based research, though not always in the same architectural sense. In closed-loop trajectory prediction, a goal-oriented, transformer-based scene decoder is used to expose an ego predictor to its own self-induced states during on-policy simulation, with reported relative collision rate reductions of up to 27.0% on nuScenes and 79.5% in dense DeepScenario intersections compared to open-loop baselines (Yadav et al., 25 Mar 2026). That work is described as “very closely related to Reactive Transformer (RxT)-style closed-loop trajectory prediction” because it shares the premise that prediction systems should remain stable when their own outputs are fed back into the environment (Yadav et al., 25 Mar 2026).

In reactive human motion generation, Transformer-based models are evaluated on paired action–reaction motion prediction from boxing interactions. The study compares a simple Transformer, iTransformer, and Crossformer, and reports that the simple Transformer can generate plausible interaction-aware motions without posture collapse, whereas iTransformer and Crossformer accumulate errors over time; it also introduces a person ID embedding to maintain structural consistency and better capture interaction dynamics (Soga et al., 24 Apr 2026). This work does not propose RxT as a named architecture, but it reinforces the broader research pattern in which “reactive” Transformer usage refers to stability under interaction, feedback, and mutual dependence rather than isolated one-shot prediction (Soga et al., 24 Apr 2026).

Taken together, these adjacent papers suggest that the conceptual core associated with RxT—statefulness, closed-loop robustness, and explicit handling of interaction dynamics—extends beyond conversational language modeling, even when the exact architecture differs. Within language modeling proper, however, RxT is specifically the event-driven, STM-based architecture introduced for stateful real-time processing of dialogue (Filipek, 3 Oct 2025).