OR-CONVQA: Open-Retrieval Conversational QA

• OR-CONVQA is a form of conversational QA that retrieves evidence from large corpora to answer context-dependent queries without relying on pre-provided passages.
• The system typically employs a retriever-reranker-reader pipeline that integrates history-aware query encoding and dense retrieval methods, including innovations like graph-guided retrieval.
• Current research advances focus on improving dynamic history modeling, mitigating error propagation from predicted answers, and enhancing domain adaptation.

Searching arXiv for recent and foundational papers on open-retrieval conversational question answering. Found papers including foundational ORConvQA, graph-guided retrieval, MICQA, ConvADR-QA, PRO-ConvQA, NORMY, QAConv, and synthetic-data/domain adaptation work. Open-retrieval conversational question answering (OR-CONVQA), also written ORConvQA, OrConvQA, or ODConvQA, is a form of conversational question answering in which a system must answer the current turn by retrieving supporting evidence from a large external collection and then extracting or generating an answer. In the foundational formulation, the input is the $k$-th question $q_k$, the preceding questions $\{q_i\}_{i=1}^{k-1}$, and a passage collection $C$, and the goal is to predict $a_k$ without assuming ground-truth history answers (Qu et al., 2020). Subsequent work characterizes the task as harder than standard open-domain QA because the current question is often elliptical, coreferential, and dependent on prior turns while the evidence is not given in advance, and harder than passage-given conversational reading comprehension because the system must search over millions of passages as well as interpret dialog context (Li et al., 2021).

1. Definition and scope

OR-CONVQA occupies the intersection of open-domain QA, conversational QA, and conversational search. Its defining property is that retrieval is part of the task rather than a preprocessing convenience. In standard conversational machine reading comprehension, the relevant passage is already supplied; in OR-CONVQA, evidence must be found in a large collection before answer prediction can begin (Qu et al., 2020). This shifts the central technical burden from pure answer extraction to a coupled problem of conversational question understanding, retrieval, reranking, and reading.

The task is frequently contrasted with reader-side conversational QA models that assume a known passage. For example, the HAE line of work formulates a training instance as $(p, q_k, \mathbf{H}_k, a_k)$, where $p$ is given in advance; from an OR-CONVQA perspective, this is a conversational reading-comprehension component rather than a full open-retrieval system (Qu et al., 2019). Conversely, some adjacent benchmarks move the “conversational” property from the user-query side to the evidence side. QAConv, for instance, studies answering standalone questions over long conversations, with retrieval in full mode bounded to a known dialogue or channel rather than unrestricted corpus-wide search (Wu et al., 2021). A plausible implication is that OR-CONVQA should be understood not as a single benchmark family, but as a systems problem whose defining feature is context-dependent querying over a large evidence space.

The field also distinguishes OR-CONVQA from response ranking and from single-turn open-domain QA. Response ranking selects from candidate replies rather than retrieving evidence and grounding an answer. Single-turn open-domain QA searches a corpus but does not usually need to resolve anaphora, ellipsis, topic continuation, topic return, or follow-up intent from preceding dialog turns (Qu et al., 2020). That distinction explains why history modeling is not an auxiliary refinement in OR-CONVQA; it is part of the retrieval problem itself.

2. Canonical architecture and context representation

The canonical OR-CONVQA architecture is a retriever-reranker-reader pipeline. In the original ORConvQA system, a dense dual-encoder retriever maps a history-aware query and corpus passages into a shared vector space, a cross-encoder reranker refines passage order, and a reader predicts answer spans with shared normalization across retrieved passages (Qu et al., 2020). The retriever score is defined as

$S_{rt}(q_k^{rt}, p_j) = v_q v_p^\top,$

with passage vectors precomputed offline for maximum inner product search (Qu et al., 2020). This pipeline became the reference point for later work, even when later systems replaced or expanded particular modules.

History representation is the second axis of design. Early ORConvQA concatenates previous questions within a fixed window and prepends the first question to the retriever input because the first turn often expresses the global information need of the dialog (Qu et al., 2020). Adjacent ConvQA work quickly showed that naïve concatenation is not the only option. HAE marks tokens in the passage that belong to selected previous answer spans and remains robust when using around 5 or 6 history turns, whereas prepending methods deteriorate when too many turns are included (Qu et al., 2019). HAM adds soft selection over prior turns and shows that different turns deserve different weights, with notable behavior under drill-down, topic shift, and topic return (Qu et al., 2019). DHS-ConvQA pushes this argument further by combining entity-based pruning, attention-based reranking, and token-level highlighting, and reports that selecting relevant turns works better than rewriting the original question in passage-grounded ConvQA (Zaib et al., 2023).

Structured conversational cues form a parallel design line. CONVSR replaces full question rewriting with compact structured representations of the form

$$\langle \text{context entities} \mid \text{question entities} \mid \text{question predicates} \mid \text{expected answer types} \rangle,$$

arguing that dependency resolution does not require generating a verbose standalone question (Zaib et al., 2023). NORMY extends the same intuition to OR-CONVQA proper by arguing that the best history is module-specific: broader context is useful for retrieval so relevant documents are not missed, whereas narrower context is better for locating an exact answer span in later stages (Rashid et al., 2024). Taken together, these lines of work suggest that “history modeling” in OR-CONVQA is not a single method but a family of choices about which prior turns, prior answers, entities, predicates, and reformulations should be visible to which module.

3. Retrieval-centered methodological progression

A striking feature of the literature is that the largest gains typically come from improvements to retrieval, retrieval conditioning, or retrieval-aware history modeling rather than from radically new answer extractors. The following methods illustrate the main progression.

Method	Core retrieval idea	Reported result
ORConvQA (Qu et al., 2020)	Learnable dense retriever with reranker and reader	OR-QuAC test F1 29.4
Graph-guided multi-round retrieval (Li et al., 2021)	Dynamic History Modeling plus hyperlink-based Explorer over passage graph	Test F1 33.4 with predicted history answers; 45.3 with true history answers
MICQA (Liang et al., 2022)	KL-regularized retriever pretraining, post-ranker, curriculum learning	Test F1 35.0, Rt Recall 0.4202
ConvADR-QA (Fang et al., 2022)	Historical-answer-aware dense retrieval with teacher-student denoising	Extractive F1 38.4, MRR@5 66.8, R@5 77.9
PRO-ConvQA (Jeong et al., 2023)	Direct phrase retrieval with conversational dependency modeling via contrastive learning	OR-QuAC F1 36.84 / EM 15.73; TopiOCQA F1 36.67 / EM 20.38
NORMY (Rashid et al., 2024)	Non-uniform history modeling plus history-aware unsupervised retriever	End-to-end F1 0.0782 on ORQUAC, 0.1625 on doc2dial-OR, 0.1723 on ConvMix

Graph-guided retrieval made the answer-history question explicit. Its key claim is that prior methods mainly modeled history through previous questions, whereas relations between previous answers and the current answer were neglected. The method seeds a local hyperlink graph with passages containing previous answers, dense retrieval results, and TF-IDF retrieval results, then propagates information with a graph attention network and performs one-hop exploration (Li et al., 2021). The empirical result that about 60% of answers can be found in two-hop connected passages containing history answers motivated this answer-aware graph view, and the ablations show that removing Explorer hurts more than removing the ranker or the dynamic history module (Li et al., 2021).

MICQA reframed the retrieval problem as one of question-understanding mismatch. It uses rewritten questions as a supervision signal during retriever pretraining and minimizes bidirectional KL divergence between the retrieval distributions induced by the rewritten question and the conversationally contextualized question (Liang et al., 2022). It then adds a post-ranker with ranking-order and distance-structure constraints, plus curriculum learning to reduce reliance on manual gold-passage insertion. The largest retrieval drop in MICQA’s ablation comes from removing the KL-regularized retriever pretraining, indicating that retrieval-side conversational understanding is the dominant contributor (Liang et al., 2022).

ConvADR-QA and related answer-aware systems argue that previous answers are not merely auxiliary context but the most compact available state of the conversation. ConvADR-QA encodes the current question together with previous question-answer pairs, uses a teacher-student scheme to align noisy conversational inputs with context-independent rewrites, and improves both extractive and generative pipelines on OR-QuAC (Fang et al., 2022). This line is conceptually close to graph-guided retrieval, but instead of activating answer-containing graph neighborhoods it injects historical answers directly into the dense retriever input.

4. Alternative formulations and evidence spaces

The standard retriever-reranker-reader decomposition is not the only formulation. PRO-ConvQA replaces passage retrieval followed by reading with direct phrase retrieval. It makes the contrast explicit: $$f(a_i \mid \mathrm{Conv}_i) = M_{\mathrm{retr}}(P_K \mid \mathrm{Conv}_i) \times M_{\mathrm{read}}(a_i \mid P_K)$$ for the conventional pipeline, versus

$q_k$0

for direct phrase retrieval (Jeong et al., 2023). The model then adds a turn-level contrastive loss that pulls adjacent conversational contexts together and pushes irrelevant ones apart. This reframing is important because it treats open-domain conversational QA as direct answer retrieval rather than document retrieval followed by extraction.

Weakly supervised OR-CONVQA addresses a different bottleneck: the mismatch between conversational answers and exact spans. The weakly supervised line observes that exact span-match weak supervision works for span answers but is unsatisfactory for freeform answers. It therefore trains a separate weak supervisor to identify paraphrased evidence spans in retrieved passages and shows that this learned weak supervision is complementary to exact span matching, especially on freeform-answer OR-CoQA (Qu et al., 2021). A plausible implication is that OR-CONVQA training regimes must be sensitive not only to corpus retrieval but also to the form of the answer supervision available.

The evidence space can also be generalized beyond plain text passages. CONVINSE studies conversational QA over heterogeneous sources—KB facts, text sentences, tables, and infoboxes—and uses a structured representation to retrieve and verbalize evidence from all sources before a fusion-in-decoder model generates the answer (Christmann et al., 2022). Its benchmark, ConvMix, comprises 3000 real-user conversations with 16000 questions, and the results show that combining sources improves both answer presence and end-to-end P@1 (Christmann et al., 2022). This is not the standard Wikipedia-passage OR-CONVQA setting, but it extends the open-retrieval logic to mixed evidence modalities.

A further neighboring formulation uses conversations themselves as the knowledge source. QAConv evaluates question answering over informative conversations such as emails, panel discussions, and work channels, with chunk mode providing the grounded chunk and full mode requiring retrieval of the relevant conversational chunk (Wu et al., 2021). The setup is bounded rather than unrestricted, but it exposes retrieval problems that standard document-oriented open QA does not. At the structured end of the spectrum, CONVEX treats follow-up conversational questions as controlled graph expansion over a knowledge graph, maintaining an evolving context subgraph and selecting “frontier” nodes for judicious expansion (Christmann et al., 2019). This is not unstructured OR-CONVQA, but it is a retrieval-intensive conversational QA formulation with closely related context-resolution issues.

5. Benchmarks, supervision, and evaluation

The benchmark ecology of OR-CONVQA reflects different assumptions about evidence, retrieval scope, and conversational dependence.

Benchmark	Scale	Distinctive role
OR-QuAC (Qu et al., 2020)	5,644 dialogs; 40,527 questions; 11M passages	Foundational OR-CONVQA benchmark derived from QuAC, CANARD, and Wikipedia
TopiOCQA (Jeong et al., 2023)	45,450 training turns; 2,514 validation turns	Open-domain conversational QA with topic switching
QReCC (Perera et al., 6 Sep 2025)	13.6K conversations; 80K QA pairs	Large-scale open-domain ConvQA combining rewriting, retrieval, and reading comprehension
ConvMix (Christmann et al., 2022)	3000 conversations; 16000 questions	Heterogeneous-source conversational QA over KB, text, tables, and infoboxes
QAConv (Wu et al., 2021)	34,608 QA pairs; 10,259 conversations	Questions over conversations as evidence; full mode retrieves chunks within a known dialogue or channel

OR-QuAC is the foundational benchmark because it made open retrieval central to the conversational setting, with 4,383/490/771 train/dev/test dialogs and 31,526/3,430/5,571 questions over an English Wikipedia corpus of about 11 million passages (Li et al., 2021). TopiOCQA introduced explicit topic switching, making it a useful counterpoint to same-topic conversational datasets (Jeong et al., 2023). QReCC systematized the rewrite-retrieve-read pipeline at scale, while ConvMix expanded the evidence space to heterogeneous sources (Perera et al., 6 Sep 2025, Christmann et al., 2022).

Supervision regimes differ sharply across datasets and models. The original ORConvQA work used full supervision with gold passages during training (Qu et al., 2020). Weakly supervised OR-CONVQA removes the assumption that gold supporting passages are available and instead trains readers with weak answer labels derived from exact span matching or a learned paraphrase-aware supervisor (Qu et al., 2021). Synthetic-data approaches push this further toward deployment. A recent pipeline starts from raw domain documents, extracts user-salient propositions, generates synthetic dialogs with both contextualized and decontextualized questions plus proposition grounding, and then uses the resulting data to train lightweight rewriters or retrievers so that existing dialog-unaware retrievers can be reused (Vlachos et al., 7 Jul 2025). In enterprise settings, this reframes OR-CONVQA as a domain-adaptation problem rather than purely a benchmark problem.

Evaluation is typically two-level. Retrieval is measured with metrics such as MRR, Recall@$q_k$1, MAP, Precision, or answer presence, while answering is measured with F1, EM, HEQ-Q, HEQ-D, or related end-to-end metrics (Rashid et al., 2024, Qu et al., 2021). The weak-supervision literature adds diagnostic measures such as % Has Answer, % Hit Gold, Recall, and % From Gold to separate failures of retrieval from failures of evidence alignment (Qu et al., 2021). This division is not incidental: it encodes a central fact about OR-CONVQA, namely that end-to-end failure can arise from missing evidence, misranked evidence, or incorrect answer extraction even when relevant evidence is present.

6. Limitations, recurring failure modes, and research directions

A recurrent misconception is that more history is always better. The history-selection literature argues the opposite. DHS-ConvQA shows that irrelevant but semantically similar turns can sharply degrade performance, especially under topic shift and topic return, and that pruning has a larger effect than later reranking in its passage-grounded setting (Zaib et al., 2023). HAM’s attention visualizations make the same point in softer form: nearby turns matter in drill-down, but remote turns can become most important under topic return, and the useful history is neither always the last turn nor always the full transcript (Qu et al., 2019). For OR-CONVQA, this suggests that history selection is a retrieval-control problem, not just a reader convenience.

A second recurring limitation is error propagation through predicted history answers. Answer-aware retrieval methods work best when previous answers are reliable, but the gap between predicted-answer and gold-answer settings remains large. Graph-guided retrieval improves test F1 from 29.4 to 33.4 with predicted history answers but from 34.6 to 45.3 with true history answers, indicating a large oracle gap (Li et al., 2021). ConvADR-QA shows a similar pattern, with substantially better retrieval and QA when gold historical answers are used rather than predicted ones (Fang et al., 2022). This suggests that answer-aware retrieval is powerful but fragile: it improves state tracking when the state is correct, and amplifies mistakes when it is not.

A third limitation is source dependence. Graph-guided retrieval relies on Wikipedia hyperlink structure and explicitly notes that transfer to corpora without rich links is nontrivial, with knowledge graphs proposed as future support (Li et al., 2021). QAConv shows that a dense retriever trained on Wikipedia transfers poorly to conversational evidence, and that BM25 can outperform transferred dense retrieval in full mode over emails, panel discussions, and Slack data (Wu et al., 2021). These findings indicate that “open retrieval” in OR-CONVQA is not a single retrieval problem: corpus structure, discourse style, and chunking assumptions strongly affect method choice.

Current research directions therefore concentrate on domain adaptation, module-specific history, and grounded generation. NORMY argues for non-uniform history modeling across retriever, reranker, and reader rather than one universal conversational encoding (Rashid et al., 2024). Synthetic-data pipelines generate decontextualized rewrites and proposition-level grounding so that lightweight models can be trained even when no human OR-CONVQA annotations exist (Vlachos et al., 7 Jul 2025). The recent survey literature frames dynamic history management, ambiguity handling, cross-domain adaptability, and retrieval-augmented grounding as central open directions for conversational QA systems more broadly (Perera et al., 6 Sep 2025). Within OR-CONVQA specifically, the cumulative literature suggests that the central unsolved issue is not merely better answer extraction; it is robust conversational retrieval under incomplete, shifting, and error-prone dialog state.