How Much Capacity Does EEG Denoising Need? Ultra-Compact Networks reveal Benchmark Saturation and Metric-Utility Gap

Abstract: Deep learning EEG denoising architectures have scaled from tens of thousands to tens of millions of parameters, yet no prior study has isolated model capacity as the experimental variable or tested whether reconstruction metrics predict downstream neural-signal utility. We address both gaps by fixing architecture, loss, data split, and training recipe while sweeping only channel width from 1.05K to 40.26K parameters in a minimal depthwise-separable convolutional U-Net. Models were evaluated on the EEGDenoiseNet benchmark, cross-dataset BCI transfer tests, controlled baseline retraining, and downstream motor-imagery classification with five decoder families across all nine BCI Competition IV-2a subjects. Reconstruction performance saturated by 3-6.5K parameters, with post-elbow gains of at most 0.015 correlation coefficient per log10-parameter unit. An 8.46M-parameter baseline retrained under the same pipeline matched the 40.26K compact variant on EOG--a 200x parameter gap yielding no advantage--while a Patch-Transformer control reproduced the same diminishing-return shape. Downstream evaluation exposed a classifier-dependent metric-utility gap: reconstruction-optimized denoising significantly degraded CSP+LDA classification across all nine subjects and three artifact types (best denoised accuracy 0.547 vs. 0.612 noisy baseline; Bonferroni p=0.0488), persisting on naturally recorded trials (Delta=-0.047; BH-FDR q=0.0049). End-to-end neural decoders showed variable or neutral effects. Standard EEG denoising benchmarks are saturated far below current model capacity, and reconstruction metrics do not predict BCI utility. Ultra-compact models at 33-46 KB and 1.27-2.61M FLOPs/segment are practical for edge deployment. These findings argue for capacity-controlled evaluation, harder task-aware benchmarks, and mandatory downstream validation.

• The paper reveals that standard EEG denoising benchmarks saturate with only 3–6.5K parameters, showing negligible improvements beyond this range.
• The paper identifies a metric–utility gap where improvements in reconstruction metrics lead to reduced downstream motor-imagery classification performance.
• The paper demonstrates that ultra-compact, parameter-efficient architectures are sufficient for effective EEG denoising in embedded applications.

Capacity Requirements for EEG Denoising: Benchmark Saturation and Limits of Metric-Based Evaluation

Overview

This paper rigorously analyzes the parameter efficiency and role of model capacity in supervised EEG denoising, questioning assumptions underlying standard evaluation protocols. Using a controlled experiment that isolates model width as the sole variable, the study demonstrates that prevailing benchmarks are saturated far below the parameter regime of contemporary high-capacity models. Furthermore, it establishes a consistent metric–utility gap: improvements in common reconstruction metrics (e.g., correlation coefficient, RMSE, SDR) do not guarantee preservation of task-relevant information, sometimes degrading downstream EEG classification performance.

Experimental Design and Methodology

A family of depthwise-separable convolutional U-Nets was constructed, with parameter counts swept from 1.05K to 40.26K by varying channel width. All other architectural and training variables were held constant. Evaluation was conducted on several fronts:

• Benchmark datasets: EEGDenoiseNet (semi-synthetic EOG/EMG benchmarks), a newly designed million-segment mixed-artifact corpus, and zero-shot transfer to BCI Competition IV datasets.
• Baselines: Architectural controls, such as retrained EEGDenoiseNet CNN, MicroWaveNet, and a Patch-Transformer capacity sweep.
• Downstream tasks: Motor-imagery classification on BCI Competition IV-2a, spanning all nine subjects and five decoder architectures, including spatial-filter pipelines (CSP+LDA) and end-to-end neural decoders.
• Resource profiling: Analysis of model size, FLOPs, CPU/GPU latency, and memory.

Key Findings

1. Benchmark Saturation with Ultra-Compact Models

Reconstruction performance—measured by CC and related metrics—saturates by 3–6.5K parameters. Increasing model size beyond this regime yields, at most, 0.015 CC gain per log10-parameter increment. The controlled 40.26K-parameter U-Net matched the performance of an 8.46M-parameter retrained EEGDN CNN on EOG artifact removal, revealing a greater than 200× parameter gap with no practical reconstruction improvement. A Patch-Transformer control sweep verified that this saturation is architectural-agnostic.

On the more challenging mixed-artifact benchmark and zero-shot BCI datasets, the same early plateau was observed. Notably, transfer performance to BCI IV-2b declined for larger models, suggesting that additional capacity overfits to training-distribution idiosyncrasies, impairing generalization.

2. Metric–Utility Gap: Degradation of Downstream Utility

Optimization solely for reconstruction metrics led to significant and systematic degradation in downstream BCI classification tasks. Specifically:

• CSP+LDA motor-imagery accuracy decreased for all denoised variants compared to noisy/noisy baselines, with the best denoised/denoised accuracy at 0.547 versus 0.612 for the baseline (Bonferroni $p=0.0488$).
• The metric–utility gap persisted across all nine BCI subjects and for multiple artifact types, as well as on real, non-synthetic EEG recordings ($\Delta$ in accuracy = –0.047, BH-FDR $q=0.0049$).
• End-to-end neural decoders exhibited variable or neutral effects, indicating utility loss is classifier-dependent.
• The amplitude suppression effect was identified as a likely culprit: denoising trained on semi-synthetic benchmarks tends to indiscriminately attenuate large-amplitude signals, erasing both artifacts and potentially task-relevant neural features.

3. Parameter-Efficient Deployment

Models in the base4–base6 width regime (3.22K–6.53K parameters, 33–46 KB, 1.27–2.61M FLOPs) are sufficient to saturate all major benchmarks. Embedded and edge EEG applications do not require large-capacity models for standard denoising tasks, and such compact models offer substantial efficiency advantages.

Implications

Practical and Theoretical Consequences

• On Benchmark Interpretation: The standard semi-synthetic benchmarks used throughout the field are insufficiently challenging and do not require modern, overparameterized deep architectures. Apparent improvements in large models typically reflect protocol artifacts or training/augmentation, not architectural advances.
• On Model Selection: Deploying high-capacity models for denoising provides negligible or negative returns. Compact architectures are preferable for practical and reproducible EEG denoising, especially in embedded settings.
• On Metric Use: Reliance on canonical reconstruction metrics is insufficient and potentially misleading. Superior metric scores may correspond to transformation of the EEG signal in ways that degrade its subsequent utility, especially for spatial-filtering decoders.
• On Pipeline Validation: Mandatory evaluation on downstream signal utility—using both linear spatial filters and end-to-end neural decoders, with both synthetic and real artifacts—is required for meaningful validation.
• On Benchmark and Loss Design: Future benchmarks must incorporate realistic, unlabeled artifacts and be coupled to task-aware evaluation endpoints. Loss functions solely targeting reconstruction metrics are inadequate; multitask or Pareto objectives that optimize both denoising fidelity and downstream neural feature preservation deserve investigation.

Future Directions

The findings motivate several avenues in signal processing and neural network design for neuroengineering:

• Task-aware Denoising: Integrating differentiable or multi-objective loss functions that reward both fidelity and downstream task preservation.
• Benchmark Reform: Developing large-scale, task-coupled benchmarks with naturalistic artifacts and explicit downstream endpoints, such as seizure or event detection.
• Model Generalization: Studying how model capacity affects robustness and transfer in more complex, multi-channel, and non-additive artifacts scenarios.
• Edge Deployment: Realizing parameter- and compute-efficient architectures as standard practice in portable and wearable EEG devices.

Conclusion

This paper demonstrates that standard EEG denoising benchmarks are saturated by ultra-compact models and that continued parameter scaling offers negligible utility. More importantly, it establishes a systematic disconnect between traditional reconstruction metrics and preservation of downstream neural information. For the field to progress, capacity-controlled methodology, harder benchmarks, and—critically—mandatory downstream validation must become routine. Future EEG denoising research should prioritize not only signal fidelity but also its impact on subsequent neural signal processing, classification, and interpretation.

Ultra-compact models should be the default for edge EEG denoising, and evaluation must extend beyond metric benchmarks to ensure preservation of neural utility.