- The paper discusses the critical importance and methods for evaluating reliability and validity in TMS-EEG biomarkers, which are essential for clinical utility.
- It highlights challenges like large off-target noise compromising internal reliability for lateral brain stimulation and questioning external reliability due to experimental variations and off-target effects, noting that the largest TEP components are often reliable but not valid.
- Recommendations for improvement include hardware optimization, individualized stimulation parameters, artifact minimization during data collection, sophisticated offline analytic methods, and increased collaboration across labs.
The paper "Reliability and validity of TMS-EEG biomarkers" (2207.08456) discusses the critical importance of reliability (both internal and external) and validity in the context of TMS-EEG biomarkers, essential for their clinical utility. The paper highlights controversies arising from the presence of large off-target components (noise) and relatively weak genuine brain responses (signal) in TMS-EEG recordings.
Key Findings on Reliability and Validity
Reliability
Initial TMS-EEG studies targeting medial structures with minimal muscle artifact demonstrated high internal reliability. However, the internal reliability can be compromised when stimulating lateral brain regions due to strong muscle artifacts and sensory responses. External reliability of TEPs has been questioned, with controversies arising from differing TEP results across labs due to variations in experimental setups and off-target effects.
Validity
The paper posits that early TEP components (<50 ms) may reflect more valid aspects of local cortical excitability, but their utility is limited by noise. Later components (>50 ms) are significantly affected by sensory non-specific effects. The largest amplitude components in the TEP are often reliable but not valid, comprising non-neural, peripherally-evoked, and off-target neural sources. MEPs are presented as a gold standard due to their high internal and external reliability and validity in tracking corticospinal excitability.
Methods for Evaluating TMS-EEG Biomarkers
The paper outlines several methods for evaluating TMS-EEG biomarkers:
- Internal Reliability: Assessed via test-retest measurements within a lab over time (minutes, hours, weeks).
- External Reliability: Evaluated through repeated quantification across different instruments or in different laboratories and comparison of different preprocessing and analytic pipelines.
- Validity (Non-invasive): Assessed by correlating TEPs with other noninvasive measures like MEPs and examining TEP changes in different neurological and psychiatric disorders.
- Validity (Invasive): Establishing links between TEPs and intracranial neurophysiology through TMS-evoked intracranial electrophysiology, electrical stimulation-evoked electrophysiology, and simultaneous invasive and noninvasive brain recordings.
Recommendations to Increase Reliability and Validity
Several recommendations are made to enhance the reliability and validity of TMS-EEG biomarkers:
- Hardware Optimization: Employing EEG amplifiers with specialized features (high sampling rate, high dynamic range, etc.) to reduce pulse artifact.
- Stimulation Parameter Individualization: Optimizing stimulation location, coil angle, and intensity in real-time to maximize the SNR of specific TEP features.
- Artifact Minimization During Data Collection: Reducing electrode impedance, using active electrodes and foam between the coil and scalp, employing passive (earmuffs) and active noise masking, and rearranging electrode wires.
- Offline Analytic Methods: Removing TMS pulse artifacts, line noise, eye movement artifacts, and off-target neural sources using interpolation, filters, and ICA; carefully considering which TEP peaks to quantify (P30, N45, P60, N100, P200) and how to quantify them (peak amplitude, area under the curve, etc.); comparing real and sham TMS statistically; and employing machine learning for data-driven approaches.
- Collaboration: Encouraging collaboration with external labs to assess external reliability and with labs using intracranial methods to assess biomarker validity.
Challenges in TMS-EEG Biomarker Research
The paper identifies several challenges:
- Off-target effects: The presence of large-amplitude, reliable off-target components (noise) in TEPs can confound the interpretation of true neural signals.
- Separating sensory responses: Difficulties in separating off-target sensory responses and non-neural artifacts from genuine brain responses.
- SNR: Low signal-to-noise ratio, particularly for early TEP components.
Future Directions
The paper suggests the following future directions:
- Standardization: Developing standardization across the TMS-EEG community with respect to reliability and validity assessments.
- Data Sharing: Promoting dissemination and data sharing.
- Intracranial methods: Encouraging investigators to initiate collaborations with labs using intracranial methods to assess biomarker validity.
- Sophisticated statistical techniques: Exploring more sophisticated statistical techniques (such as machine learning methods) for feature extraction from TEP data.
In summary, the paper emphasizes the necessity of evaluating and optimizing both the reliability and validity of TMS-EEG biomarkers. It provides methods for assessment and offers recommendations to improve these critical metrics, highlighting current challenges and suggesting future directions for the field, including standardization, data sharing, and the application of sophisticated statistical techniques.
