Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 56 tok/s
Gemini 2.5 Pro 38 tok/s Pro
GPT-5 Medium 26 tok/s Pro
GPT-5 High 22 tok/s Pro
GPT-4o 84 tok/s Pro
Kimi K2 182 tok/s Pro
GPT OSS 120B 420 tok/s Pro
Claude Sonnet 4.5 30 tok/s Pro
2000 character limit reached

Query Augmentation by Decoding Semantics from Brain Signals (2402.15708v2)

Published 24 Feb 2024 in cs.CL, cs.AI, and cs.IR

Abstract: Query augmentation is a crucial technique for refining semantically imprecise queries. Traditionally, query augmentation relies on extracting information from initially retrieved, potentially relevant documents. If the quality of the initially retrieved documents is low, then the effectiveness of query augmentation would be limited as well. We propose Brain-Aug, which enhances a query by incorporating semantic information decoded from brain signals. BrainAug generates the continuation of the original query with a prompt constructed with brain signal information and a ranking-oriented inference approach. Experimental results on fMRI (functional magnetic resonance imaging) datasets show that Brain-Aug produces semantically more accurate queries, leading to improved document ranking performance. Such improvement brought by brain signals is particularly notable for ambiguous queries.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. Hervé Abdi and Lynne J. Williams. 2010. Principal component analysis. Wiley Interdisciplinary Reviews: Computational Statistics, 2(4):433–459.
  2. Context attentive document ranking and query suggestion. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 385–394.
  3. When relevance judgement is happening? An EEG-based study. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 719–722.
  4. A comparison of human brainwaves-based biometric authentication systems. International Journal of Biometrics, 12(4):411–429.
  5. MS MARCO: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268.
  6. Relevance prediction from eye-movements using semi-interpretable convolutional neural networks. In Proceedings of the 2020 Conference on Human Information Interaction and Retrieval, pages 223–233.
  7. Iterative relevance feedback for answer passage retrieval with passage-level semantic match. In Advances in Information Retrieval: 41st European Conference on IR Research, ECIR 2019, Cologne, Germany, April 14–18, 2019, Proceedings, Part I 41, pages 558–572. Springer.
  8. David Carmel and Elad Yom-Tov. 2010. Estimating the query difficulty for information retrieval. Morgan & Claypool Publishers.
  9. A hybrid framework for session context modeling. ACM Transactions on Information Systems, 39(3):1–35.
  10. Web search via an efficient and effective brain-machine interface. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pages 1569–1572.
  11. Predicting query performance. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 299–306.
  12. Decoding speech perception from non-invasive brain recordings. Nature Machine Intelligence, pages 1–11.
  13. Natural brain-information interfaces: Recommending information by relevance inferred from human brain signals. Scientific Reports, 6(1):38580.
  14. Predicting term-relevance from brain signals. In Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 425–434.
  15. Kunihiko Fukushima. 1980. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36(4):193–202.
  16. Temporal dynamics of eye-tracking and eeg during reading and relevance decisions. Journal of the Association for Information Science and Technology, 68(10):2299–2312.
  17. Improved query difficulty prediction for the web. In Proceedings of the 17th ACM conference on Information and knowledge management, pages 439–448.
  18. Language is not all you need: Aligning perception with language models. arXiv preprint arXiv:2302.14045.
  19. Unsupervised dense information retrieval with contrastive learning. arXiv preprint arXiv:2112.09118.
  20. Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 20(4):422–446.
  21. Kalervo Järvelin and Jaana Kekäläinen. 2017. IR evaluation methods for retrieving highly relevant documents. In ACM SIGIR Forum, volume 51, pages 243–250. ACM New York, NY, USA.
  22. A query log analysis of dataset search. In Web Engineering: 17th International Conference, ICWE 2017, Rome, Italy, June 5-8, 2017, Proceedings 17, pages 429–436. Springer.
  23. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
  24. Victor Lavrenko and W Bruce Croft. 2017. Relevance-based language models. In ACM SIGIR Forum, volume 51, pages 260–267. ACM New York, NY, USA.
  25. A natural language fMRI dataset for voxelwise encoding models. Scientific Data, 10(1):555.
  26. Latent retrieval for weakly supervised open domain question answering. arXiv preprint arXiv:1906.00300.
  27. Systematic review automation tools for end-to-end query formulation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2141–2144.
  28. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81.
  29. Visual instruction tuning. arXiv preprint arXiv:2304.08485.
  30. Gpt understands, too. AI Open.
  31. Cogtaskonomy: Cognitively inspired task taxonomy is beneficial to transfer learning in NLP. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 904–920.
  32. Fine-tuning LLaMA for multi-stage text retrieval. arXiv preprint arXiv:2310.08319.
  33. Query suggestion using hitting time. In Proceedings of the 17th ACM Conference on Information and Knowledge Management, pages 469–478.
  34. Clara Meister and Ryan Cotterell. 2021. Language model evaluation beyond perplexity. arXiv preprint arXiv:2106.00085.
  35. Query performance prediction: From ad-hoc to conversational search. arXiv preprint arXiv:2305.10923.
  36. Understanding feeling-of-knowing in information search: An EEG study. ACM Transactions on Information Systems, 42(3):1–30.
  37. Predicting human brain activity associated with the meanings of nouns. science, 320(5880):1191–1195.
  38. Understanding information need: An fMRI study. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, pages 335–344.
  39. Javed Mostafa and Jacek Gwizdka. 2016. Deepening the role of the user: Neuro-physiological evidence as a basis for studying and improving search. In Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval, pages 63–70.
  40. Broca’s area and the language instinct. Nature Neuroscience, 6(7):774–781.
  41. The “Narratives” fMRI dataset for evaluating models of naturalistic language comprehension. Scientific Data, 8(1):250.
  42. Toward a universal decoder of linguistic meaning from brain activation. Nature Communications, 9(1):963.
  43. Iterative learning to rank from explicit relevance feedback. In Proceedings of the 35th Annual ACM Symposium on Applied Computing, pages 698–705.
  44. The cortical activity of graded relevance. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 299–308.
  45. Stephen Robertson. 2004. Understanding inverse document frequency: on theoretical arguments for idf. Journal of Documentation, 60(5):503–520.
  46. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends in Information Retrieval, 3(4):333–389.
  47. Joseph John Rocchio Jr. 1971. Relevance feedback in information retrieval. The SMART Retrieval System: Experiments in Automatic Document Processing.
  48. Predicting query performance by query-drift estimation. ACM Transactions on Information Systems, 30(2):1–35.
  49. Semantic reconstruction of continuous language from non-invasive brain recordings. Nature Neuroscience, pages 1–9.
  50. BEIR: A heterogenous benchmark for zero-shot evaluation of information retrieval models. arXiv preprint arXiv:2104.08663.
  51. LLaMA: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  52. Attention is all you need. Advances in neural information processing systems, 30.
  53. Zhenhailong Wang and Heng Ji. 2022. Open vocabulary electroencephalography-to-text decoding and zero-shot sentiment classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 5350–5358.
  54. Unicorn: Unified cognitive signal reconstruction bridging cognitive signals and human language. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13277–13291.
  55. Quantifying query ambiguity with topic distributions. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pages 1877–1880.
  56. Language generation from human brain activities. arXiv preprint arXiv:2311.09889.
  57. Relevance feedback with brain signals. ACM Transactions on Information Systems, 42(4):Article No. 93.
  58. Brain topography adaptive network for satisfaction modeling in interactive information access system. In Proceedings of the 30th ACM International Conference on Multimedia, pages 90–100.
  59. Towards a better understanding of human reading comprehension with brain signals. In Proceedings of the ACM Web Conference 2022, pages 380–391.
  60. Why don’t you click: Understanding non-click results in web search with brain signals. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 633–645.
  61. Deep learning for brain disorder diagnosis based on fmri images. Neurocomputing, 469:332–345.
  62. Towards brain-to-text generation: Neural decoding with pre-trained encoder-decoder models. In NeurIPS 2021 AI for Science Workshop.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

This paper has been mentioned in 1 post and received 8 likes.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up to View

Don't miss out on important new AI/ML research

See which papers are being discussed right now on X, Reddit, and more:

Explore Trending Papers

“Emergent Mind helps me see which AI papers have caught fire online.”

Philip

Philip

Creator, AI Explained on YouTube