Impact of LLM-driven Conversational Retrieval on System Performance

Ascertain whether and to what extent large language model-driven conversational retrieval methods affect lifelog retrieval system performance in the ACM Lifelog Search Challenge across task types.

Background

LSC'24 saw growing adoption of LLMs and conversational interfaces (e.g., GPT-3.5 Turbo, Mistral7B, and RAG pipelines). Despite this trend, several top-performing systems were not explicitly reliant on conversational LLMs.

The authors explicitly state that the impact of LLM-driven conversational retrieval on system performance is unclear, motivating an investigation into its effectiveness relative to other factors such as embedding-based retrieval and interface design.

References

Although LLM-driven conversational retrieval methods have been increasingly adopted, their impact on system performance remains unclear.

— The State-of-the-Art in Lifelog Retrieval: A Review of Progress at the ACM Lifelog Search Challenge Workshop 2022-24 (2506.06743 - Tran et al., 7 Jun 2025) in Section 4.4 Insights on Techniques and System Performance, Subsection Large Language Models and Conversational Search