2000 character limit reached
RAICL: Retrieval-Augmented In-Context Learning for Vision-Language-Model Based EEG Seizure Detection
Published 25 Jan 2026 in cs.HC, cs.AI, and cs.LG | (2601.17844v1)
Abstract: Electroencephalogram (EEG) decoding is a critical component of medical diagnostics, rehabilitation engineering, and brain-computer interfaces. However, contemporary decoding methodologies remain heavily dependent on task-specific datasets to train specialized neural network architectures. Consequently, limited data availability impedes the development of generalizable large brain decoding models. In this work, we propose a paradigm shift from conventional signal-based decoding by leveraging large-scale vision-LLMs (VLMs) to analyze EEG waveform plots. By converting multivariate EEG signals into stacked waveform images and integrating neuroscience domain expertise into textual prompts, we demonstrate that foundational VLMs can effectively differentiate between different patterns in the human brain. To address the inherent non-stationarity of EEG signals, we introduce a Retrieval-Augmented In-Context Learning (RAICL) approach, which dynamically selects the most representative and relevant few-shot examples to condition the autoregressive outputs of the VLM. Experiments on EEG-based seizure detection indicate that state-of-the-art VLMs under RAICL achieved better or comparable performance with traditional time series based approaches. These findings suggest a new direction in physiological signal processing that effectively bridges the modalities of vision, language, and neural activities. Furthermore, the utilization of off-the-shelf VLMs, without the need for retraining or downstream architecture construction, offers a readily deployable solution for clinical applications.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
Simple explanation of the paper
1) What is this paper about?
The paper is about a new way to detect epileptic seizures from brain signals using AI. Instead of training special models on raw signal data, the authors turn the brain signals into pictures and ask a powerful “vision-and-language” AI (a model that can look at images and read text) to decide if a short clip contains a seizure. They also show the AI a few smartly chosen example images to help it make better decisions on the fly.
2) What questions did the researchers ask?
In simple terms, they asked:
- Can we turn EEG brain signals (the wavy lines measured by sensors on your head) into pictures and let a modern AI that “sees” and “reads” understand them?
- If we give the AI a few helpful examples that are carefully chosen, can it detect seizures as well as, or better than, traditional methods?
- Can we do this without retraining the big AI model, so it’s easy to use in real clinics?
3) How did they do it? (Methods in everyday language)
Think of an EEG as a set of squiggly lines that show brain activity over time—like a heart monitor, but for your brain. Here’s their approach:
- Turn signals into pictures:
- They stack the squiggly lines from many EEG channels into one clean image, using different colors for each line so they don’t blend together.
- They draw the lines clearly (thicker strokes, no clutter) so the AI can “see” important details.
- Teach the AI what to look for with words:
- They write a short, step-by-step set of instructions that explains what seizure patterns look like (for example, sudden spikes, rhythms, or synchronized changes).
- This “Chain-of-Thought” style prompt encourages the AI to explain its reasoning before giving an answer.
- Show the AI a few helpful examples:
- Their key idea, called RAICL (Retrieval-Augmented In-Context Learning), is like giving the AI a mini “study guide” for each new case.
- For each new EEG picture to classify, they automatically pick a few example images:
- Non-seizure examples come from the same person’s past calm brain activity (so the AI learns that person’s normal baseline).
- Seizure examples come from other patients’ data, chosen to look most similar to the new case.
- To choose these examples, they use a “visual fingerprint” from a pre-trained image encoder to measure which examples are most representative (typical) and most similar to the current case.
- Ask a vision-LLM (VLM) to decide:
- They feed the example images, the new EEG image, and the written instructions into a VLM (like Gemini-3-Flash, or open-source models such as Qwen3-VL or InternVL).
- The VLM reasons step by step and outputs whether the clip contains a seizure or not.
They tested this on two hospital datasets of infants and newborns with seizures. Importantly, they didn’t retrain the big AI; they just used smart prompts and example selection.
4) What did they find, and why does it matter?
Main findings:
- Turning EEG signals into clear, colored waveform pictures works surprisingly well. A standard vision model (like ResNet) trained on these images performed strongly—sometimes better than specialized EEG signal models.
- Giving the AI domain hints (the step-by-step text instructions) and a few examples improved accuracy.
- Their RAICL method for picking the best examples gave a big, consistent boost over just using random examples.
- The best results came from a state-of-the-art VLM (Gemini-3-Flash) combined with RAICL. It matched or beat strong traditional methods without retraining the big model.
- Open-source VLMs did well but didn’t quite match the top proprietary model. Results suggest the visual part of the AI (the image encoder) is especially important for this task.
- Using different colors for the channels helped the AI separate overlapping lines and understand the image better than black-and-white plots.
Why it matters:
- EEG analysis often needs a lot of specific training data and can be hard to adapt to new patients. This method works in a “zero-training” way: turn EEG into images, add good examples, and ask a powerful VLM.
- It could make seizure detection more flexible and easier to deploy in hospitals, especially when patient-specific seizure data is limited.
5) What could this change in the future? (Impact and next steps)
- A new direction: bridging vision, language, and brain signals. Instead of building complex, specialized EEG models, we can tap into general-purpose vision-LLMs with smart prompts and selected examples.
- Faster clinical use: Because this approach doesn’t require retraining the big AI model, it could be quicker to put into real-world practice.
- Better tools ahead: Since the visual encoder seems to matter most, future work can build EEG-aware image encoders and pair them with lighter LLMs for faster, cheaper use.
- Beyond seizures: The same idea could help with other brain-signal tasks, like identifying types of seizures or other neurological events, as these models and prompts improve.
Knowledge Gaps
Knowledge gaps, limitations, and open questions
The following points summarize what remains missing, uncertain, or unexplored in the paper, and are framed to be concrete and actionable for future research.
- Scope limited to binary seizure vs. non-seizure classification on short (4 s) windows; no evaluation of multi-class tasks (e.g., seizure subtypes), event-level onset/offset detection, or sequence-level post-processing.
- Clinical relevance metrics (e.g., sensitivity, specificity, ROC-AUC, false alarm rate per hour, detection latency, time-to-first-detection) are not reported; balanced accuracy alone is insufficient for clinical deployment.
- Window-level classification is evaluated, but event-level aggregation, smoothing, and false alarm management strategies are not explored.
- Downsampling of datasets prior to VLM inference (retaining every tenth trial) may introduce selection bias and makes cross-method comparisons uneven; the impact on performance and fairness of comparison needs quantification.
- Reliance on a proprietary API (Gemini-3-Flash) raises reproducibility and privacy concerns; compute cost, throughput, latency, and context window constraints for real-time/bedside deployment are unreported.
- RAICL assumes availability of test subject’s historical non-task trials; sensitivity to the quantity, recency, and quality of these anchors, and fallback strategies when no history exists, are not studied.
- RAICL uses CLIP as a fixed visual encoder for retrieval; the impact of using each VLM’s native encoder or domain-tuned encoders for retrieval embeddings is not evaluated.
- Example selection relies on cosine distance and medoids; alternative retrieval metrics (e.g., learned similarity, Mahalanobis distance), clustering strategies, or cross-subject normalization are not compared.
- Number of shots fixed at M=2; trade-offs between more/fewer in-context examples, saturation points with larger context windows, and memory/performance scaling are unexplored.
- Prompt design and Chain-of-Thought (CoT) content are hand-crafted; sensitivity to wording, length, structure, and systematic prompt optimization (e.g., automated prompt search, instruction tuning) is not analyzed.
- CoT explanations are not validated for clinical faithfulness (e.g., alignment with expert criteria, hallucination rate, and explanation fidelity); no clinician-in-the-loop evaluation.
- Comparisons with image-based alternatives (spectrograms/topomaps) are not performed under the same VLM framework; it remains unclear which EEG-to-image representation is best per task.
- Waveform plotting hyperparameters (scaling α, offset δ, color palette, stroke thickness, resolution, margins) are motivated but not quantitatively ablated across datasets and VLMs.
- Robustness to artifacts (EOG/EMG/motion), extreme amplitudes, electrode detachment, and channel/montage mismatches is not characterized; cross-device and cross-montage generalization is untested.
- Generalization across age groups, adult populations, pathologies, and acquisition setups (sampling rates, electrode layouts) beyond pediatric/neonatal data remains unknown.
- No statistical significance testing, confidence intervals, or per-subject paired comparisons are provided for VLM vs. baselines; viability of reported gains under statistical scrutiny is uncertain.
- Open-source VLMs underperform relative to Gemini, but the root causes are not analyzed; disentangling visual encoder quality vs. LLM head, and testing encoder swaps or lightweight fine-tuning, is left open.
- The claim that performance is “mostly driven by the visual encoder” is plausible but not rigorously validated via controlled ablations (e.g., same encoder with different LLMs, or same LLM with different encoders).
- Zero-training paradigm is emphasized; potential gains from modest domain-adaptive fine-tuning (e.g., visual encoder adapters, LoRA) and the trade-offs (compute, sample size) are not examined.
- Only 4-second non-overlapping windows are used; performance sensitivity to window length, overlap, and multi-scale temporal contexts is not evaluated.
- Calibration of output probabilities (e.g., reliability diagrams, ECE) and decision thresholding tailored to clinical risk profiles are not addressed.
- Failure mode analysis (e.g., confusion patterns, patient-level errors, seizure morphology missed, non-seizure states misclassified) is absent; actionable error characterization is needed.
- Integration of textual retrieval (e.g., guidelines, patient metadata) alongside visual example retrieval is unexplored; multimodal retrieval policies could be evaluated.
- Real-time deployment aspects (on-device inference, memory footprint, energy consumption, throughput) and edge/embedded feasibility are not assessed.
- Privacy/security considerations for transmitting patient EEG images to cloud APIs (HIPAA/GDPR compliance, de-identification protocols) are not discussed.
- Impact of label noise and annotation uncertainty (especially in NICU consensus labels) on RAICL selection and VLM decisions is not quantified; robustness to noisy labels should be tested.
- Dataset size and class imbalance handling in the VLM setting (given downsampling) are not examined; effects on retrieval diversity and example representativeness need study.
- Combining RAICL with temporal priors or sequential models (e.g., HMMs, CRFs) for smoother predictions over continuous recordings is not explored.
- Fairness and subgroup analyses (sex, age, clinical condition) and potential performance disparities are not reported.
- Preprocessing specifics (filters, artifact removal, referencing schemes) are brief; sensitivity of VLM performance to preprocessing choices and reproducibility across pipelines is untested.
Glossary
- Amplitude clipping: A plotting technique that limits waveform amplitude to prevent overlap, potentially at the cost of signal fidelity. Example: "Alternatively, amplitude clipping could also handle overlap, but is suboptimal as it compromises signal integrity."
- Autoregressive: A generation paradigm where a model predicts the next token conditioned on previously generated tokens. Example: "condition the autoregressive outputs of the VLM."
- Balanced Classification Accuracy (BCA): An evaluation metric that averages per-class accuracy to handle class imbalance. Example: "Balanced classification accuracy (BCA) was used as the metric for evaluation"
- Bipolar channels: An EEG montage formed by subtracting adjacent electrode signals to emphasize local activity. Example: "split into 4-second non-overlapping trials with bipolar channels"
- Chain-of-Thought (CoT): A prompting strategy that guides models to produce intermediate reasoning steps before final answers. Example: "strategies like Chain-of-Thought (CoT) enhance reasoning by incorporating intermediate logical steps into these exemplars"
- Centroid: The mean vector of embeddings representing the central tendency of a class or set. Example: "we compute the test subject's non-task centroid in the embedding space:"
- Chromatic Encoding: Assigning distinct colors to channels to visually disentangle overlapping signals. Example: "Chromatic Encoding. Signal amplitudes often exhibit significant variance, leading to potential overlap between adjacent channels."
- Common Spatial Patterns (CSP): A spatial filtering method for EEG used to maximize variance differences between classes. Example: "common spatial patterns for motor imagery sensorimotor function decoding"
- Cosine distance: A distance metric based on the cosine of the angle between two vectors. Example: "We define the distance metric d(·, ·) as the cosine distance:"
- Differential entropy: A continuous analog of entropy used as a statistical feature in signal analysis. Example: "differential entropy for emotion recognition"
- Event-Related Potential (ERP): Brain responses time-locked to specific events or stimuli used in EEG analysis. Example: "xDAWN for event-related potential visual decoding"
- Few-shot: A learning setup in which models are conditioned with only a small number of labeled examples. Example: "most representative and relevant few-shot examples"
- Foundation model: A large, general-purpose model intended to be adapted across tasks and domains. Example: "developing a foundation model for brain decoding"
- High-Fidelity Rendering: Plotting practices that preserve fine-grained waveform morphology for accurate visual encoding. Example: "High-Fidelity Rendering Fine-grained morphological features, such as transient high-frequency oscillations or sharp waveform inflections, are critical diagnostic events."
- In-Context Learning (ICL): Adapting model behavior by providing examples directly in the prompt without parameter updates. Example: "In-Context Learning (ICL) enables models to adapt to new tasks by embedding input-output pairs directly into the inference prompt"
- Leave-one-subject-out cross-validation: A validation scheme where one subject is held out for testing while others are used for training. Example: "we used a standard leave-one-subject-out cross-validation."
- Medoid: The representative sample of a set minimizing average distance to other points, used as a prototype. Example: "we employ a nearest medoid strategy"
- Median absolute deviation: A robust measure of variability used for amplitude normalization. Example: "(e.g., median absolute deviation)"
- Non-stationarity: The property of signals whose statistical characteristics change over time. Example: "To address the inherent non-stationarity of EEG signals"
- Projector: A component that maps visual embeddings into the LLM’s token space. Example: "Certain VLM architectures utilize a projector to map these embeddings into the LLM's token space."
- Representativeness (metric): A measure of how prototypical a sample is with respect to its class centroid. Example: "We then define the representativeness metric, m_rep, as the distance to its class centroid:"
- Resting-state: Baseline EEG recorded without task engagement, used as non-task anchors. Example: "resting-state trials"
- Retrieval-Augmented In-Context Learning (RAICL): A method that selects and prepends relevant examples to improve in-context reasoning. Example: "we introduce a Retrieval-Augmented In-Context Learning (RAICL) approach"
- Similarity (metric): A measure of closeness between a prototype and a query, often using cosine distance. Example: "The relevance of each prototype is scored based on its similarity to the query of representation:"
- t-SNE: A dimensionality reduction technique for visualizing high-dimensional embeddings. Example: "-SNE visualization of RAICL selection strategies, with both representativeness + similarity, given (two-shot) examples."
- Time-Frequency Spectrogram: A visualization of signal energy across time and frequency. Example: "Time-Frequency Spectrogram: A 2D heatmap representing spectral energy distribution over time and frequency"
- Topographical Map: A 2D spatial projection of EEG potentials or power across the scalp. Example: "Topographical Map: A spatial projection of voltage or power onto a 2D circular plane within certain duration"
- Visual encoder: The component that converts images into high-dimensional embeddings for multimodal models. Example: "forwarded to the VLM's visual encoder."
- Vision-LLM (VLM): A model that integrates visual and textual inputs for joint reasoning. Example: "Vision-LLMs (VLMs) offers a promising alternative."
- Visual embeddings: Vector representations of images used to condition LLMs. Example: "serving as visual embeddings that contrast with or align with the test trial embeddings' patterns"
- Waveform Plot: A visualization of EEG voltage over time, optionally stacked across channels. Example: "Waveform Plot: Provides a continuous temporal fluctuation of electrical potentials at the scalp surface with time on the horizontal axis and voltage amplitude on the vertical axis"
- Zero-shot: A setting where the model encounters classes with no labeled examples from the target subject. Example: "A cross-subject, task-zero-shot setting is considered"
Practical Applications
Immediate Applications
Below is a curated set of applications that can be deployed with minimal additional development, leveraging the paper’s findings on converting EEG to stacked waveform images, RAICL-based example selection, and zero-training use of off-the-shelf vision-LLMs.
- Clinical seizure detection assistant for NICU/pediatric monitoring (healthcare)
- Description: Deploy VLM-based, zero-calibration seizure detection as a decision support tool in neonatal and pediatric ICUs. The system ingests live EEG, renders stacked chromatic waveform plots, retrieves representative non-seizure anchors from the patient’s own history and similar seizure prototypes from auxiliary subjects (RAICL), and produces a classification plus a Chain-of-Thought (CoT) rationale for clinician review.
- Tools/products/workflow: EEG-to-image rendering library (high-fidelity, chromatic encoding, stacked channels); RAICL module using CLIP embeddings; prompt templates with diagnostic criteria and output constraints; API integration to a proprietary VLM (e.g., Gemini-3-Flash) or on-prem open-source VLM (Qwen3-VL, InternVL).
- Assumptions/dependencies: Availability of short historical non-seizure data for each patient; EEG channel montage consistency; acceptable latency (API/edge); HIPAA/GDPR compliance; human-in-the-loop review; performance consistent with reported BCA (≈70–82% avg in datasets).
- Retrospective EEG triage and pre-annotation (healthcare, academia)
- Description: Rapidly triage long EEG recordings to flag candidate seizure segments and provide explainable summaries to speed expert annotation.
- Tools/products/workflow: Batch waveform plotting; RAICL retrieval tuned for retrospective queries; CoT rationales exported into annotation systems.
- Assumptions/dependencies: Access to auxiliary labeled datasets for seizure exemplars; variability across sites handled by visualization standardization (color coding, stroke width, resolution).
- Quality control and artifact flagging in EEG acquisition (healthcare, software)
- Description: Use VLM reasoning to identify channel overlap, clipping, electrode detachment, motion artifacts, and montage inconsistencies by inspecting stacked waveform images.
- Tools/products/workflow: CoT prompts tailored to artifact taxonomies; chromatic encoding to surface cross-channel overlap; dashboard warnings for technicians.
- Assumptions/dependencies: Reliable rendering pipeline; domain prompts capturing artifact signatures; acceptance of a decision-support, not diagnostic, role.
- Explainable training aids for neurodiagnostics (education)
- Description: Use CoT prompts that encode diagnostic criteria to teach trainees how to interpret EEG seizure patterns via model-generated step-by-step rationales tied to electrode identities.
- Tools/products/workflow: Prompt library; curated exemplar bank (medoids by class and subject); integration with teaching platforms.
- Assumptions/dependencies: Stable VLM outputs (temperature set to 0); supervision from expert faculty to correct model errors; consistent plotting standards across cases.
- Cross-site deployment without retraining (healthcare, industry)
- Description: Roll out the zero-training RAICL workflow across hospitals or clinics using existing EEG systems; only the visualization and retrieval modules are installed, with VLM inference done via API or local GPU.
- Tools/products/workflow: Vendor-agnostic middleware (EEG-to-VLM); MLOps hooks to monitor VLM drift and version changes; audit logging of CoT rationales.
- Assumptions/dependencies: Contracted API SLAs or on-prem GPU capacity; site-specific data governance approvals; harmonized electrode labels in waveform images.
- Accelerated dataset curation and benchmarking (academia)
- Description: Use RAICL+VLM to bootstrap labels and hard-negative mining; generate explainable rationales to improve inter-rater agreement and reduce curation time.
- Tools/products/workflow: Batch inference pipelines; retrieval index (per class/per subject medoids); curator review interfaces.
- Assumptions/dependencies: Clear acceptance criteria for auto-label confidence; institutional IRB approval for semi-automated labeling; continuous evaluation against gold standards.
- Integration guidance for device manufacturers (industry)
- Description: Embed the paper’s plotting design choices (chromatic per-channel encoding, high-fidelity rendering, stacked layout) and RAICL selection in EEG device software to improve downstream AI comprehension.
- Tools/products/workflow: Firmware/UI updates; on-device CLIP embeddings; configurable prompt templates; options for edge or cloud inference.
- Assumptions/dependencies: Regulatory classification as clinical decision support; performance monitoring; user controls for prompt variations.
Long-Term Applications
The following applications require further clinical validation, scaling, specialized encoders, or regulatory pathways before broad deployment.
- Real-time, on-device seizure monitoring with regulatory approval (healthcare, industry)
- Description: Edge deployment of an optimized visual encoder aligned with a lightweight LLM to meet latency, power, and reliability requirements for continuous monitoring.
- Tools/products/workflow: Specialized visual encoder fine-tuned for EEG images; model compression/quantization; redundant inference on-device; FDA/CE approval workflows.
- Assumptions/dependencies: Robustness across montages and patient populations; longitudinal clinical trials; cybersecurity and safety certifications.
- Multi-class seizure subtype identification and early warning (healthcare)
- Description: Extend from binary detection to subtype classification (e.g., focal vs generalized), and toward pre-ictal early warning using RAICL with larger context windows and richer exemplar banks.
- Tools/products/workflow: Expanded prompt schemas for subtype criteria; larger retrieval indices; continuous monitoring pipelines; alert prioritization logic.
- Assumptions/dependencies: More labeled subtype data; stronger visual encoders; validated sensitivity/specificity for early warning to minimize false alarms.
- Closed-loop neurostimulation and adaptive therapy (healthcare, robotics)
- Description: Use fast on-device detection as a trigger for neurostimulation systems to abort seizures in responsive neurostimulation or VNS setups.
- Tools/products/workflow: Low-latency inference modules; safety interlocks; clinician-programmable prompt constraints; A/B testing against current detection systems.
- Assumptions/dependencies: Millisecond-level latency; extremely low false positives; rigorous safety cases and regulatory approvals.
- Generalizable brain decoding for other EEG tasks (academia, healthcare, consumer wellness)
- Description: Apply the EEG-to-image + RAICL paradigm to sleep staging, event-related potentials (e.g., P300), emotion recognition, and motor imagery; develop task-specific visual encoders that outperform generic CV backbones.
- Tools/products/workflow: Task-tailored plotting (e.g., spectrograms or topographies when appropriate), retrieval strategies per task, specialized encoders aligned to lightweight LLMs.
- Assumptions/dependencies: Adequate labeled datasets per task; domain prompts capturing task-specific criteria; evaluation on cross-subject/zero-shot settings.
- Cross-modal extension to other physiological signals (healthcare, software)
- Description: Port the approach to ECG, EMG, and multimodal biosignals by rendering stacked waveform images and leveraging RAICL to adapt across subjects.
- Tools/products/workflow: Signal-specific plotting standards; exemplar banks for arrhythmias/muscle disorders; shared visual encoders across modalities.
- Assumptions/dependencies: Clinical validation per modality; careful prompt engineering for modality-specific features; buy-in from professional societies on visualization standards.
- Telemedicine and home monitoring at scale (healthcare, daily life)
- Description: Continuous home EEG monitoring with cloud-based RAICL inference and clinician dashboards; caregiver alerts with explainable summaries.
- Tools/products/workflow: Dry-cap EEG devices; secure data pipelines; cloud VLM services with retrieval indices; mobile apps for caregivers.
- Assumptions/dependencies: Reliable home-grade EEG; robust artifact handling; reimbursement frameworks; clear escalation protocols for alerts.
- Standardization and policy frameworks for multimodal clinical AI (policy, healthcare)
- Description: Develop standards for EEG visualization (color schemes, resolution, label overlays), retrieval practices (patient anchoring, medoid selection), explainability requirements, and performance reporting for VLM-based clinical tools.
- Tools/products/workflow: Consensus guidelines with neurology societies; regulatory guidance documents; benchmarking suites; compliance checklists.
- Assumptions/dependencies: Multi-stakeholder collaboration; alignment with privacy and medical device regulations; openness to auditing black-box APIs.
- MLOps for VLM-based clinical decision support (software, policy)
- Description: Operationalize monitoring of model drift, prompt versioning, RAICL index updates, and auditability of CoT rationales in hospital IT environments.
- Tools/products/workflow: Prompt registries; retrieval index lifecycle management; inference logging; bias and robustness dashboards.
- Assumptions/dependencies: Integration with hospital IT; secure and compliant data storage; governance for model updates (e.g., API provider changes).
- Educational simulators and interactive tutoring (education)
- Description: Build interactive systems that let students probe EEG cases, modify prompts, and see how RAICL selection alters reasoning and outcomes.
- Tools/products/workflow: Case libraries; prompt-edit UIs; visualization overlays explaining channel-electrode mapping and feature salience.
- Assumptions/dependencies: Institution licenses for VLMs; faculty-curated content; safeguards preventing overreliance on model outputs.
- “EEG-to-VLM” middleware products (industry, software)
- Description: Commercial libraries/services that standardize plotting, RAICL retrieval, prompt construction, and model-agnostic inference for healthcare vendors.
- Tools/products/workflow: SDKs for device integration; model adapters for proprietary/open VLMs; prebuilt clinical prompts and retrieval policies.
- Assumptions/dependencies: Market acceptance; maintenance of compatibility with evolving VLMs; robust customer support and validation kits.
Collections
Sign up for free to add this paper to one or more collections.