Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Alljoined1 -- A dataset for EEG-to-Image decoding (2404.05553v3)

Published 8 Apr 2024 in q-bio.NC and cs.AI

Abstract: We present Alljoined1, a dataset built specifically for EEG-to-Image decoding. Recognizing that an extensive and unbiased sampling of neural responses to visual stimuli is crucial for image reconstruction efforts, we collected data from 8 participants looking at 10,000 natural images each. We have currently gathered 46,080 epochs of brain responses recorded with a 64-channel EEG headset. The dataset combines response-based stimulus timing, repetition between blocks and sessions, and diverse image classes with the goal of improving signal quality. For transparency, we also provide data quality scores. We publicly release the dataset and all code at https://linktr.ee/alljoined1.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Faster ICA under orthogonal constraint. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4464–4468. IEEE, 2018.
  2. Visual image reconstruction based on EEG signals using a generative adversarial and deep fuzzy neural network. Biomedical Signal Processing and Control, 87:105497, 2024.
  3. Confounds in the Data—Comments on “Decoding Brain Representations by Multimodal Learning of Neural Activity and Visual Features”. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):9217–9220, 2022. Conference Name: IEEE Transactions on Pattern Analysis and Machine Intelligence.
  4. A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence. Nature neuroscience, 25(1):116–126, 2022.
  5. Dreamdiffusion: Generating high-quality images from brain eeg signals. arXiv preprint arXiv:2306.16934, 2023.
  6. Brain decoding: toward real-time reconstruction of visual perception. In The Twelfth International Conference on Learning Representations, 2024.
  7. Machine learning of brain-specific biomarkers from EEG. bioRxiv, 2024.
  8. BOLD5000, a public fMRI dataset while viewing 5000 visual images. Scientific data, 6(1):49, 2019.
  9. Structure-Preserved Image Reconstruction from Brain Recordings. In preparation, 2023.
  10. Cinematic mindscapes: High-quality video reconstruction from brain activity. Advances in Neural Information Processing Systems, 36, 2024.
  11. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  12. Differential temporal dynamics during visual imagery and perception. Elife, 7:e33904, 2018.
  13. A large and rich EEG dataset for modeling human visual object recognition. NeuroImage, 264:119754, 2022.
  14. MEG and EEG data analysis with MNE-Python. Frontiers in neuroscience, 7:70133, 2013.
  15. The representational dynamics of visual objects in rapid serial visual processing streams. NeuroImage, 188:668–679, 2019.
  16. Human EEG recordings for 1,854 concepts presented in rapid serial visual presentation streams. Scientific Data, 9(1):3, 2022.
  17. The temporal dynamics of scene processing: A multifaceted EEG investigation. Eneuro, 3(5), 2016.
  18. Independent component analysis, adaptive and learning systems for signal processing, communications, and control. John Wiley & Sons, Inc, 1:11–14, 2001.
  19. Autoreject: Automated artifact rejection for MEG and EEG data. NeuroImage, 159:417–429, 2017.
  20. MOABB: trustworthy algorithm benchmarking for BCIs. Journal of neural engineering, 15(6):066011, 2018.
  21. Tomoyasu Horikawa & Yukiyasu Kamitani. Generic decoding of seen and imagined objects using hierarchical visual features. Nature Communications, 2017.
  22. Brain2Image: Converting Brain Signals into Images. In Proceedings of the 25th ACM international conference on Multimedia, pages 1809–1817, Mountain View California USA, 2017. ACM.
  23. Visual Saliency and Image Reconstruction from EEG Signals via an Effective Geometric Deep Network-Based Generative Adversarial Network. Electronics, 11(21):3637, 2022. Number: 21 Publisher: Multidisciplinary Digital Publishing Institute.
  24. Encoding and Decoding Framework to Uncover the Algorithms of Cognition. In The Cognitive Neurosciences. The MIT Press, 2020.
  25. Yixuan Ku. Selective attention on representations in working memory: cognitive and neural mechanisms. PeerJ, 6:e4585, 2018.
  26. Seeing through the Brain: Image Reconstruction of Visual Perception from Human Brain Signals, 2023. arXiv:2308.02510 [cs, eess, q-bio].
  27. Brain2pix: Fully convolutional naturalistic video reconstruction from brain activity. BioRxiv, pages 2021–02, 2021.
  28. Training on the test set? An analysis of Spampinato et al. [31], 2018. arXiv:1812.07697 [cs, q-bio].
  29. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer, 2014a.
  30. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer, 2014b.
  31. Standardized measurement error: A universal metric of data quality for averaged event-related potentials. Psychophysiology, 58(6):e13793, 2021.
  32. NeuroGAN: image reconstruction from EEG signals via an attention-based GAN. Neural Computing and Applications, 35(12):9181–9192, 2023.
  33. The neural dynamics of facial identity processing: insights from EEG-based pattern analysis and image reconstruction. Eneuro, 5(1), 2018.
  34. A multivariate investigation of visual word, face, and ensemble processing: Perspectives from EEG-based decoding and feature selection. Psychophysiology, 57(3):e13511, 2020.
  35. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PMLR, 2021.
  36. Deep learning-based electroencephalography analysis: a systematic review. Journal of Neural Engineering, 16(5):051001, 2019.
  37. Reconstructing the Mind’s Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
  38. MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data. arXiv preprint arXiv:2403.11207, 2024.
  39. EEG2IMAGE: Image Reconstruction from EEG Brain Signals, 2023. arXiv:2302.10121 [cs, q-bio].
  40. Learning Robust Deep Visual Representations from EEG Brain Recordings. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 7553–7562, 2024.
  41. Deep learning human mind for automated visual classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6809–6817, 2017.
  42. On high-pass filter artifacts (they’re real) and baseline correction (it’sa good idea) in ERP/ERMF analysis. Journal of neuroscience methods, 266:166–170, 2016.
  43. Gated transformer for decoding human brain EEG signals. In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pages 125–130. IEEE, 2021.
  44. Multidimensional object properties are dynamically represented in the human brain. bioRxiv, 2023.
  45. Speed of processing in the human visual system. nature, 381(6582):520–522, 1996.
  46. ThoughtViz: Visualizing Human Thoughts Using Generative Adversarial Network. In Proceedings of the 26th ACM international conference on Multimedia, pages 950–958, Seoul Republic of Korea, 2018. ACM.
  47. MindBigData 2022 A Large Dataset of Brain Signals. arXiv preprint arXiv:2212.14746, 2022.
  48. Photorealistic Reconstruction of Visual Texture From EEG Signals. Frontiers in Computational Neuroscience, 15, 2021.
  49. Using goal-driven deep learning models to understand sensory cortex. Nature neuroscience, 19(3):356–365, 2016.
Citations (2)
View on Semantic Scholar

Summary

  • The paper introduces a novel EEG dataset recording 46,080 epochs from eight participants to enhance image reconstruction research.
  • It employs tailored stimulus presentation and rigorous preprocessing to boost signal quality and achieve a high signal-to-noise ratio.
  • The study’s comprehensive dataset supports advanced EEG-to-image decoding applications in real-time BCIs and clinical diagnostics.

Overview of "Alljoined - A dataset for EEG-to-Image decoding"

The paper "Alljoined - A dataset for EEG-to-Image decoding" introduces a comprehensive dataset tailored for EEG-to-image decoding applications. This dataset, named Alljoined, addresses several limitations inherent in previous EEG-to-image datasets and is designed to facilitate robust and generalizable image reconstruction efforts. The dataset is significant in the context of cognitive neuroscience and brain-computer interface (BCI) research, offering unprecedented insights into how the human brain encodes and processes visual information.

Key Dataset Features

The authors have compiled data from eight participants, each exposed to 10,000 natural images, resulting in a total of 46,080 epochs of brain responses recorded via a 64-channel EEG headset. The stimuli, derived from the MS-COCO dataset, were presented in a manner to maximize the signal-to-noise ratio (SNR). This was achieved through carefully designed trial durations, session and block repetitions, and a broad array of image classes. The dataset comprises detailed data quality scores to ensure transparency and reliability of the information.

Methodological Contributions

Several methodological innovations are central to the dataset:

  1. Tailored Stimulus Presentation: The trial duration and repetitions within and between blocks are specifically designed to enhance the SNR.
  2. Diverse Image Set: The inclusion of 9,000 unique naturalistic images per participant and 1,000 shared images among participants ensures a wide variety of stimuli, critical for generalizable image decoding.
  3. Qualitative Comparisons: Comparisons against existing datasets underscore the superior design and potential of Alljoined for advancing EEG-based image reconstruction.

Comparative Analysis

In comparison with existing datasets like Brain2Image and ThoughtViz, Alljoined addresses several biases and limitations. Brain2Image, for instance, has been criticized for its acquisition design, which inadvertently boosts model performance by introducing extraneous proxy information. The more diverse and extensive stimuli used in Alljoined reduce the risk of block-specific correlations and enhance the dataset's generalizability. Moreover, Alljoined's design mitigates the classification rather than reconstruction phenomena observed in datasets with limited classes.

Signal Quality and Data Processing

The EEG data underwent rigorous preprocessing, including band-pass filtering, ICA for artifact removal, and baseline correction. These steps were carefully selected to preserve the integrity of the neural signals. Noteworthy is the use of the MNE-Python library for preprocessing, ensuring that state-of-the-art methods are employed.

Analysis of Results

The authors provide extensive analyses of the recorded EEG data, including event-related potentials (ERPs) and SNR metrics. Consistent neural activity patterns were observed across individuals, with strong ERP signals peaking between 250 and 300 ms post-stimulus. The SNR analysis highlights the importance of repeated measures across sessions for accurate signal quality assessment.

Implications and Future Directions

Practically, the dataset could serve as a cornerstone for advancements in real-time BCIs and clinical diagnostic tools. The portability and cost-effectiveness of EEG make it preferable over fMRI for such applications. Theoretically, the dataset enriches the understanding of brain dynamics, particularly in how visual information is processed in real-time. Future research directions proposed include high-density EEG recordings focusing on occipital and parietal regions and exploring the generalization to imagined mental imagery.

Conclusion

Alljoined represents a substantial contribution to the domain of EEG-to-image decoding by addressing significant limitations of previous datasets and offering a robust, generalizable dataset. The dataset's potential applications span from enhancing our understanding of visual processing mechanisms to practical BCI implementations. The authors' future work promises to further extend the applicability and utility of this dataset in various cognitive and clinical settings.

Data and Code Availability

The Alljoined dataset, along with the necessary code for stimuli and preprocessing, is made publicly available to promote transparency and facilitate further research in the field. Researchers can access these resources via designated OSF and GitHub links.

By providing a well-documented, high-quality dataset, this paper establishes a new standard for EEG-to-image decoding research, likely spurring subsequent studies and innovative applications within the domain.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown