Impact of LLM-driven Conversational Retrieval on System Performance

Ascertain whether and to what extent large language model-driven conversational retrieval methods affect lifelog retrieval system performance in the ACM Lifelog Search Challenge across task types.

Background

LSC'24 saw growing adoption of LLMs and conversational interfaces (e.g., GPT-3.5 Turbo, Mistral7B, and RAG pipelines). Despite this trend, several top-performing systems were not explicitly reliant on conversational LLMs.

The authors explicitly state that the impact of LLM-driven conversational retrieval on system performance is unclear, motivating an investigation into its effectiveness relative to other factors such as embedding-based retrieval and interface design.

References

Although LLM-driven conversational retrieval methods have been increasingly adopted, their impact on system performance remains unclear.

— The State-of-the-Art in Lifelog Retrieval: A Review of Progress at the ACM Lifelog Search Challenge Workshop 2022-24  (2506.06743 - Tran et al., 7 Jun 2025) in Section 4.4 Insights on Techniques and System Performance, Subsection Large Language Models and Conversational Search

We do not have an explanation for this inversion and leave it to future work.

— RALI@iKAT 2026: Retrieval-aware User Profile Denoising via Personalized Conversational Dense Retrieval  (2610.17673 - Hui et al., 13 Oct 2026) in Section 6, subsection “Preliminary Analysis,” paragraph “Response quality is decoupled from retrieval”