Streaming & Evolving QA

• Streaming and Continually Evolving QA is defined as systems that dynamically integrate new, multimodal data streams to update answers in real time.
• Key methodologies include hybrid parametric/semi-parametric models, dynamic memory architectures, and readiness-aware modules to mitigate catastrophic forgetting.
• Empirical findings highlight the importance of temporal reasoning and proactive decision-making, demonstrating significant gains in adaptation and coherence across evolving domains.

Streaming and Continually Evolving QA encompasses a research frontier where question answering (QA) systems—across text, video, and multimodal domains—are required to adapt to dynamic, temporally unfolding data streams and evolving external knowledge. These systems must not only process and integrate new information in real time, but also maintain coherence, avoid catastrophic forgetting, and, increasingly, decide when their evidence is sufficient to answer a question, rather than simply what the answer is. This area brings together advances in continual learning, memory architectures, temporal reasoning, proactive decision-making, and hybrid parametric/semi-parametric model design.

1. Problem Definition and Core Dimensions

Streaming and evolving QA extends static QA paradigms by requiring models to handle input data (text, video, audio, or multimodal streams) arriving sequentially over long time horizons. The core challenges are:

• Adaptation to Evolving Knowledge: Models must acquire and integrate new facts (news, events, visual frames) unavailable at pretraining, updating predictions and internal representations accordingly (Liška et al., 2022).
• Temporal Causality and Memory: Systems must perform temporal reasoning, linking current and past observations, and updating internal memories while controlling information retention and relevance (Hu et al., 29 Oct 2025, Azad et al., 9 Mar 2026).
• Real-Time or Readiness-Aware Inference: In many settings, the timing of responses affects correctness—answering too early (before evidence is available) or too late (after an event) incurs explicit penalties (Azad et al., 9 Mar 2026, Lu et al., 30 Jan 2026).
• Continual Domain Shift: Streaming QA in video (VidQA) or real-world settings often involves incoming data from new domains, unseen during training, necessitating robust domain adaptation or generalization mechanisms (Cheng et al., 2024).
• Multi-turn Coherence: For multi-step or dialogue QA, models are evaluated on chains of interrelated questions/answers, with temporal or causal linkages spanning long contextual windows (Yang et al., 15 Feb 2025).

These requirements span textual open-domain QA (news, Wikipedia), multimodal human–computer interaction (live streaming commerce, mobile assistants), and embodied or egocentric vision (robotics, AR/VR glasses).

2. Dataset Construction and Benchmark Design

A diverse suite of benchmarks defines the state-of-the-art research landscape for streaming and continually evolving QA:

• StreamingQA provides nearly 150k temporally anchored train/dev questions (2007–2020), generated and human-authored, over 11M time-stamped news articles. The streaming protocol assesses adaptation by limiting the model context to articles published before each quarterly evaluation, with metrics for adaptation gap and catastrophic forgetting (Liška et al., 2022).
• SVBench offers 49,979 QA pairs across 1,353 videos, constructed via semi-automated LLM pipelines and rigorous human refinement, capturing temporal multi-turn dialogues with explicit temporal linkages across consecutive video clips (Yang et al., 15 Feb 2025).
• PhoStream targets real-world mobile assistants via 5,572 open-ended QA pairs from 578 videos—including UI walkthroughs and egocentric scenes—indexed by Instant, Backward, and Forward (proactive) temporal tasks. The evaluation pipeline includes strict response timing and an LLM-as-a-judge metric (Lu et al., 30 Jan 2026).
• StreamingCoT focuses on temporally evolving VideoQA, constructing per-second dense captions, dynamically fused into semantic segments, and augmented with chain-of-thought (CoT) annotations that include spatiotemporally grounded reasoning steps validated by humans (Hu et al., 29 Oct 2025).
• ProReady-QA supports the evaluation of readiness-aware QA—annotating evidence windows for when answers become supportable—enabling assessment of both answer correctness and timing via the ARS metric (Azad et al., 9 Mar 2026).
• StreamGaze introduces 8,521 gaze-aligned QA pairs with egocentric videos, probing models' ability to use real-time human gaze signals for temporal and proactive reasoning (Lee et al., 1 Dec 2025).

Across these datasets, key annotation strategies include hierarchical segmentation (scene or event-based), temporal alignment of questions with evidence spans, automated QA generation filtered by human reviewers, and rich meta-data for multi-turn linkages and reasoning chains.

3. Model Architectures and Adaptation Mechanisms

While traditional QA models operate in a static, fully observable regime, streaming and continual QA models implement architectural innovations for sequential adaptation, memory management, and real-time timing. Notable approaches include:

• Hybrid Parametric/Semi-Parametric QA: StreamingQA contrasts parametric (closed-book, weight-based) and semi-parametric (open-book, retrieval-augmented) adaptation. Parametric models (e.g., Transformer-XL) are updated monthly via fine-tuning or retraining to integrate new textual knowledge, with adaptation gap and forgetting closely tracked. Open-book models (e.g., FiD-RAG) rapidly adapt to new data by index updates, but require generator retraining for full benefit; updating only the retriever is suboptimal for recent knowledge (Liška et al., 2022).
• Dynamic Adapter Merging (DAM): For rehearsal-free, domain-incremental continual VidQA, DAM attaches dataset-specific adapters to a frozen backbone, then merges their weights at inference using a probability-weighted sum calculated by a non-parametric router. This setup mitigates catastrophic forgetting, enables cross-domain sharing, and gracefully handles test-time domain uncertainty (Cheng et al., 2024).
• Hierarchical and Key–Value Memory Architectures: Systems such as Click-to-Ask maintain event-level key–value memories with recency-weighted pruning, using segment embeddings to enable efficient retrieval and history-aware QA in online broadcast scenarios. This design supports persistent adaptation, concept drift management, and low-latency response (Yu et al., 19 Mar 2026).
• Readiness-Aware and Timing-sensitive Reasoning: StreamReady introduces a lightweight readiness module with a dedicated <RDY> token and MLP head, trained to predict answer readiness using a contrastive loss against temporally aligned evidence. The model triggers answers only when sufficient support is observed, achieving high effective accuracy on benchmarks with explicit evidence windows (Azad et al., 9 Mar 2026).
• Scene- and Event-Segmented Storage: Vista, for streaming video QA, dynamically segments video streams into visually coherent scenes, compresses scenes into dense tokens, and indexes them for efficient retrieval and re-integration into model context upon arbitrary post-hoc queries, enabling scalable long-context video reasoning at constant GPU memory (Lu et al., 9 Feb 2026).
• Multi-level Embodied Reasoning: StreamEQA and StreamingCoT distinctly categorize tasks according to perception, interaction, and planning (for embodied agents), and implement hierarchical context windows and explicit object-state transition tracking in reasoning chains (Wang et al., 4 Dec 2025, Hu et al., 29 Oct 2025).

4. Evaluation Protocols and Metrics

Evaluation in streaming and continual QA departs from static QA by incorporating temporal alignment, memory durability, and chain-level coherence:

• Adaptation and Forgetting: StreamingQA tracks adaptation gap (Δ_adapt) and forgetting (Δ_forget) to quantify the trade-off between new knowledge assimilation and retention of older information (Liška et al., 2022).
• Temporal and Streaming Metrics: ProReady-QA and StreamReady define the Answer Readiness Score (ARS) to penalize both early and late responses, multiplying correctness by readiness for an effective accuracy. PhoStream employs a strict online protocol: only answers provided within proscribed "evidence windows" are valid, directly scoring for correct timing; failure modes include Early Response and No Response (Azad et al., 9 Mar 2026, Lu et al., 30 Jan 2026).
• Multi-turn and Memory-rich Benchmarks: SVBench uses multi-dimensional metrics—semantic accuracy, contextual coherence, logical consistency, temporal understanding, and informational completeness—aggregated per QA turn, alongside latency and memory footprint. This setup assesses not just answer correctness but long-horizon dialogue management (Yang et al., 15 Feb 2025).
• Reasoning Quality and Grounding: StreamingCoT evaluates both segment-level answer accuracy and chain-of-thought fidelity—percentage of reasoning chains with all steps grounded in actual frames and objects (Hu et al., 29 Oct 2025).
• Embodied Hierarchy and Gaze Awareness: StreamEQA stratifies accuracy by reasoning level (perception, interaction, planning), while StreamGaze separately reports performance on gaze-guided past, present, and proactive tasks, highlighting substantial model-human performance gaps (Wang et al., 4 Dec 2025, Lee et al., 1 Dec 2025).

5. Empirical Findings and Model Behaviors

Experimental results consistently identify sharp contrasts between streaming and static QA, and across adaptation strategies:

• Knowledge Adaptation Trade-offs: Incremental fine-tuning in parametric QA models enables adaptation, but lags retraining and mostly aids high-frequency entities; semi-parametric index updates are fast but require generator retuning for maximal gains, especially for frequent entities (Liška et al., 2022).
• Memory and Timing: Memory-limited architectures (fixed GPU RAM, constant cache windows) such as Vista and StreamReady support scalability but demand careful design for relevance and completeness. Readiness-aware models achieve substantial gains in effective accuracy by addressing the "when to answer" problem, outperforming prior models on proactive, evidence-tied QA (Azad et al., 9 Mar 2026, Lu et al., 9 Feb 2026).
• Temporal Asymmetry: Mobile streaming QA (PhoStream) reveals that models excel at instant/backward tasks but fundamentally struggle with proactive (forward) tasks, predominantly due to premature answering; temporal asymmetry scores quantify this performance gap, reaching 64–76 points in favor of instant tasks (Lu et al., 30 Jan 2026).
• Continual Adaptation and Forgetting: Adapter-based merging in DAM reduces catastrophic forgetting by 1.9% and increases average accuracy by 9% compared to prompt-based continual learning (Cheng et al., 2024).
• Chain-of-Thought and Embodied Planning: CoT-enhanced models improve sequential and state-duration reasoning by up to +10% accuracy, but remain 35–40 points behind human on logical coherence and temporal causality tasks (Hu et al., 29 Oct 2025).
• Gaze-Awareness: Incorporating human gaze signals does not yet resolve streaming model limitations in attention tracking or proactive alerting—no current prompting or integration method closes the model-human gap exposed by StreamGaze (Lee et al., 1 Dec 2025).

A summary (selected results):

System/Metric	Adaptation Accuracy	Forgetting	ARS/effective Acc	Temporal Asymmetry	Human-Model Gap
StreamingQA (parametric)	+4–5 F1 by FT	≤5 F1 pts	N/A	N/A	Human F1 76 vs 57 OB
DAM continual VidQA	+9.1% over SOTA	−1.9%	N/A	N/A	—
StreamReady (ProReady-QA)	56.4%	N/A	ARS 0.69 / Accₑ 0.53	N/A	—
PhoStream (mobile QA)	80.83–77.18 (I/B)	N/A	N/A	64–76 pts	—
StreamGaze (gaze-augmented)	<0.54 model, 0.83 human	—	N/A	—	25–40 pts

6. Open Problems and Research Directions

Despite progress, several foundational challenges remain:

• When to Answer: Readiness estimation remains hard, especially in the presence of ambiguous cues and noisy environments. Designing architectures that couple evidence aggregation with dynamic decision thresholds (as in StreamReady) is ongoing (Azad et al., 9 Mar 2026).
• Scalable Continual Learning: Adaptive mechanisms for handling unbounded memory, concept drift, and efficient update (e.g., adapter merging, few-shot layer adaptation, time-aware retrievers) are underexplored at full, real-world scale (Cheng et al., 2024, Liška et al., 2022).
• Temporal Reasoning and Causality: Chain-of-thought approaches, segment-wise object/state tracing, and temporal linkage mining enable progress, but models remain brittle on complex planning and anticipation tasks, particularly outside of perception-level questions (Hu et al., 29 Oct 2025, Wang et al., 4 Dec 2025).
• Gaze and Attention Integration: Explicit encoding of human attentional patterns and intention (via gaze or pointer) is necessary for AR, robotics, and assistive applications; current LLMs underutilize such signals (Lee et al., 1 Dec 2025).
• Evaluation Frameworks: Metric development—combining correctness, readiness, and long-horizon coherence—together with large, well-annotated benchmarks (esp. in mobile, embodied, and proactive scenarios) is crucial (Lu et al., 30 Jan 2026).

7. Future Outlook

Streaming and continually evolving QA is moving toward unified architectures capable of reasoning over heterogeneous, temporally structured input under real-world constraints. Promising directions include:

• Hybrid models integrating parametric updates, retrieval expansion, and event/scene-based memory representations (Liška et al., 2022, Lu et al., 9 Feb 2026).
• Unsupervised and self-supervised continual adaptation: automatic memory management, drift detection, and memory decay learning (Yu et al., 19 Mar 2026).
• Proactive reasoning and meta-decision learning: joint optimization of answer content and timing, possibly via RL-based policy heads (Azad et al., 9 Mar 2026, Lu et al., 30 Jan 2026).
• Interactive and lifelong learning agents: real-time adaptation, cross-session memory sharing, and context-sensitive response in both text and multimodal domains.
• Enhanced reasoning support: methods for explicit causal, spatial, and procedural deduction, making reasoning steps auditable and self-correcting (Hu et al., 29 Oct 2025, Wang et al., 4 Dec 2025).

The field is characterized by large, publicly available datasets and toolkits (e.g., StreamingQA, SVBench, StreamingCoT), enabling rapid progress while exposing entrenched limitations of current architectures in temporal integration, memory, and readiness-aware decision-making. Streaming and continually evolving QA thus constitutes a foundational challenge and proving ground for research in robust, adaptive, and interactive AI systems.