EEG-SpikeAgent: Agentic Closed-Loop Program Synthesis for Automated EEG Spike Detection

Abstract: Automated detection of interictal epileptiform discharges in scalp electroencephalography (EEG) is clinically important, but recent high-performing deep-learning models often trade interpretability for accuracy. We introduce EEG-SpikeAgent, a closed-loop program-synthesis framework that uses a LLM agentic system to generate signal-processing features for spike detection in scalp EEG. The system iteratively proposes one deterministic EEG feature module at a time, executes the resulting code on EEG to generate tabular features, evaluates performance via a tabular classifier, summarizes run-level metrics, and feeds structured diagnostics back to the model for refinement. Across iterations, EEG-SpikeAgent proposes and refines candidate signal features and decision rules informed by model performance. We evaluated EEG-SpikeAgent on VEPISET, a public 29-channel dataset of 4-second epochs containing 2,516 discharge-containing and 22,933 non-discharge epochs. Across five-fold cross-validation with a gradient-boosted tree classifier, agent-generated features achieved an area under the receiver operating characteristic curve of 0.935, balanced accuracy of 0.699, F1 score of 0.557, sensitivity of 0.401, and specificity of 0.996 at the default operating point. At an operating point with sensitivity 0.80, mean precision was 0.470 and mean specificity was 0.900. Artifact-aware feature generation improved balanced accuracy and F1 score over spike-only feature search. These results indicate that LLM-based program synthesis can automate EEG feature engineering in auditable and inspectable code-driven manner for clinical and methodological review.

• The paper introduces an LLM-driven closed-loop system that autonomously synthesizes deterministic, inspectable EEG features and evaluates them with XGBoost on VEPISET data.
• The generated feature portfolio achieved 0.935 AUROC, 0.699 balanced accuracy, and 0.996 specificity, but default-threshold sensitivity remained limited at 0.401.
• Artifact-aware feature searches improved balanced accuracy by about 1.4 percentage points and F1 by 2.3 points, supporting transparent detector development while highlighting gaps versus deep-learning baselines.

Motivation and positioning

Automated detection of interictal epileptiform discharges (IEDs) in scalp EEG presents a persistent trade-off between accuracy and transparency. Deep-learning detectors have reached expert-level discrimination, but their latent representations complicate debugging under distribution shift, montage variation, and artifact regimes. Conversely, physics-based, feature-engineered pipelines are interpretable and data-efficient but evolve through slow manual iteration. EEG-SpikeAgent addresses this gap by framing IED feature engineering as a design-search problem delegated to an LLM agent operating in a closed loop, so that the resulting detector consists entirely of named, deterministic signal-processing code that can be inspected, ablated, and traced to classifier behavior.

System design

The system casts IED detection as binary classification over fixed 4-second multi-channel epochs $X \in \mathbb{R}^{C \times N}$ from the public VEPISET dataset (29 channels at 500 Hz; 2,516 spike and 22,933 non-spike epochs). Each agentic iteration comprises two LLM-mediated phases followed by host-side evaluation:

• Proposal phase: the model receives a prompt context containing the current code summary, dataset assumptions, performance history, and active feature manifest, and must propose exactly one complementary feature change.
• Apply phase: a code-editing model implements the plan as a minimal Python module. Guardrails restrict edits to feature_*.py modules (one per iteration), with core engine, plugin, DSL, and evaluation code held read-only.
• Host evaluation: the executor is registered, tabularized incrementally per-feature block, and evaluated with an XGBoost classifier under stratified cross-validation on the training partition. Balanced accuracy serves as the primary optimization signal given class imbalance; AUROC, F1, sensitivity, specificity, PRAUC, and confusion matrices are logged each iteration and fed back as structured diagnostics.

Feature proposals are constrained toward short-window, multiple-instance summaries rather than full-epoch averages, robust normalization rather than absolute voltage thresholds, and compact cross-channel aggregations rather than high-dimensional per-sample representations. Executors are deterministic and parameter-free; all learning is deferred to the downstream tabular classifier, cleanly separating feature discovery from statistical fitting. Artifact-focused proposals were scheduled at iterations 5, 10, and 15 to test whether explicit artifact modeling improves IED classification.

The generated features span eight families: short-window spike morphology (line length, curvature/sharpness, Teager energy), multiscale spike-wave morphology (matched-filter responses, spike-to-slow-wave coupling), focal spatial contrast, bipolar phase-reversal evidence, regional/laterality summaries, artifact-reference coupling (ear/ECG/EMG correlation), channel-quality and burst artifacts, and multiple-instance aggregation (max, p95/p99, top-$k$ means).

Detection performance

Across five-fold cross-validation with independent XGBoost classifiers (300 rounds, depth 4, learning rate 0.05), the final portfolio of roughly 400–500 agent-generated features achieved:

Metric	Mean ± SD	95% CI
AUROC	0.935 ± 0.008	0.925–0.946
Balanced accuracy	0.699 ± 0.016	0.678–0.719
F1	0.557 ± 0.034	0.516–0.599
Sensitivity	0.401 ± 0.014	0.361–0.441
Specificity	0.996 ± 0.001	0.994–0.997

At the default threshold, the classifier strongly prioritizes false-positive suppression over sensitivity — specificity of 0.996 comes at the cost of sensitivity of only 0.401. At a post-hoc operating point of 80% sensitivity, mean precision was 0.470 and mean specificity 0.900. Performance increased across iterations and plateaued after roughly iteration 15–16, with balanced accuracy leveling near 0.70 and AUROC near 0.93.

Artifact-aware ablation

Explicitly scheduling artifact-detection iterations improved balanced accuracy by approximately 1.4 percentage points and F1 by 2.3 points relative to spike-only search, driven primarily by reduced false negatives for true spikes contaminated by artifacts. Gain-based importance analysis reinforces this: although artifact features constituted only 18% of the final set, several (e.g., frontal RMS ratio, cardiac rhythmicity metrics) ranked within the top thirty. The authors caution that gain-based importance is unsigned and biased toward features offering more split opportunities, so these rankings should be read as screening analyses rather than causal claims about individual features.

Comparison to deep-learning baselines

Against prior VEPISET results, the comparison is unfavorable on raw operating-point metrics. Lin et al.'s VGG network reported 74.2% precision and 97% specificity at 80% sensitivity on an 80:20 split, and vEpiNet trained on independent data reported 63.6% precision and 95.0% specificity at the same sensitivity. EEG-SpikeAgent's 47.0% precision and 90.0% specificity at 80% sensitivity fall below both. The authors explicitly concede this shortfall and position the contribution as complementary: an auditable, inspectable feature-search procedure rather than a state-of-the-art detector.

Limitations and open questions

The paper is candid about its constraints. Sensitivity of 0.401 at the default threshold reflects absent calibration and threshold selection in the loop; incorporating a priori operating-point specification into detector design remains open. Evaluation is restricted to curated 4-second VEPISET epochs with all IED subtypes collapsed into a binary label, leaving continuous-record screening, event localization, subtype-stratified performance, and site-level distribution shift untested. Inspectability of named computations does not guarantee interpretability or clinical validity — individual high-importance features still require review against EEG examples and failure cases. Feature stability across acquisition systems, patient populations, and recording environments has not been established. The authors accordingly frame the study as proof of concept for agent-assisted feature engineering, not validation of a clinically deployable spike detector.

Conclusion

EEG-SpikeAgent demonstrates that an LLM-driven closed loop can autonomously synthesize hundreds of deterministic, clinically structured EEG features without human intervention, achieving strong discrimination (AUROC 0.935) while retaining full code-level auditability. Its operating-point performance does not match optimized deep-learning baselines, and its sensitivity at the default threshold is limited, but the framework establishes a reproducible middle ground between manual symbolic pipeline design and opaque end-to-end learning, with natural extensions to seizure detection, sleep staging, and artifact annotation.