Ventriloquism After-Effect and Auditory Recalibration

• Ventriloquism after-effect is a phenomenon where sustained exposure to discrepant audiovisual cues induces a persistent shift in unisensory auditory localization.
• The approach employs network theory, EEG spectral analysis, and multivariate decoding to reveal early auditory recalibration and lateral biases in neural processing.
• Results underscore transient neural dynamics that decay post-training, highlighting the relevance of auditory cortex adaptation and cross-modal integration.

Searching arXiv for the specified paper and closely related work to ground the article. I’m checking arXiv metadata for the named paper and related ventriloquism after-effect work. The ventriloquism after-effect is a form of cross-modal recalibration in which repeated exposure to spatially discrepant but synchronous auditory and visual stimuli produces a subsequent bias in unisensory auditory localization toward the previously presented visual location. In the formulation analyzed in "Analysis of the Ventriloquism Aftereffect Using Network Theory Techniques" (Saha, 16 Jan 2026), the phenomenon is operationalized as a persistent shift in post-training localization of sound alone after audiovisual exposure in which vision had previously biased auditory spatial perception. The work treats the after-effect as a probe of multisensory integration, auditory spatial recalibration, and the time course of neural changes induced by audiovisual discrepancy, with particular emphasis on the questions of where and when recalibration occurs and how long it persists after adaptation.

1. Conceptual definition and scope

Within the framework adopted in (Saha, 16 Jan 2026), the ventriloquism after-effect is distinguished from the online ventriloquist illusion itself. During bimodal exposure, vision captures or biases sound location perception; after sustained exposure, a unisensory auditory localization shift remains. That residual shift is the after-effect.

The central claim is not primarily a new behavioral demonstration, but a computational and neurophysiological characterization of recalibration. The principal conclusions are that recalibration occurs early in the auditory processing pathway and that the ventriloquism after-effect decays over time after exposure. The convergence of functional EEG networks, non-stationary time series analysis, and multivariate pattern classification is used to argue that post-training auditory processing differs from pre-training processing, that the strongest differences occur relatively early after sound onset, and that the effect is strongest for lateralized auditory conditions that are behaviorally most relevant to adaptation (Saha, 16 Jan 2026).

The paper situates the phenomenon within broader accounts of multisensory processing. It reviews the modality specificity hypothesis, according to which the more reliable modality dominates perception, and Bayesian integration, in which audiovisual cues are combined approximately optimally with reliability-based weighting. The findings are presented as especially compatible with a Bayesian/recalibration perspective, in which repeated cue discrepancy updates auditory spatial representation. The thesis also contrasts sensory accounts, which posit early changes in sensory representation, with cognitive or decisional accounts, which attribute bias to later stages; the reported timing favors the former. In its conclusion, the work further suggests a possible relation to predictive coding, in which internal spatial models are updated by cross-modal prediction errors, although this interpretation is explicitly not directly tested (Saha, 16 Jan 2026).

2. Experimental paradigm and operationalization

The EEG data were drawn from an experiment by Bruns, Liebnau, and Röder (2011), titled Cross-modal training induces changes in spatial representations early in the auditory processing pathway. Participants performed an auditory localization task before and after ventriloquism training, and were divided into left-adapted and right-adapted groups. The exact number of participants is not reported in the provided thesis text.

The pre- and post-training task consisted of 4 sessions with 114 trials per session. In each trial, a 2000 Hz auditory beep was presented from one of three speakers at $-10^\circ$, $0^\circ$, or $+10^\circ$ relative to the participant’s head. After a random delay of 700–900 ms, a go signal was presented from all three speakers, and participants reported perceived sound location as left, center, or right. The delayed response was introduced to reduce contamination of EEG by motor-related artifacts during the auditory evoked processing interval. Within each session, sounds from the three locations were presented equally often and in random order (Saha, 16 Jan 2026).

The adaptation procedure used synchronous light + sound pairs with a constant spatial discrepancy. The auditory component was identical to the task sound and came from the same three locations. The visual component was a yellow LED displaced by $15^\circ$ relative to the sound. In the left-adapted group, the light was $15^\circ$ to the left of the sound; in the right-adapted group, it was $15^\circ$ to the right. Training stimuli were presented in sets of five consecutive stimuli from the same location, at a rate of 1 per second, with equal numbers from each location. To ensure attention during training, participants responded to occasional deviants, either a red LED or a 1000 Hz tone (Saha, 16 Jan 2026).

The behavioral logic is straightforward: participants localized sounds before adaptation, underwent audiovisual discrepancy training, and then localized sounds again after adaptation. The expected after-effect was a post-training bias in auditory localization toward the previously experienced visual offset. Neural differences were therefore interpreted by comparing pre-training auditory trials and post-training auditory trials at the same physical sound location, effectively contrasting a sound with its “ventriloquized counterpart.” A notable pattern emphasized in the classification results is that effects were stronger for the lateral conditions opposite the adaptation direction: right-adapted subjects showed stronger effects for audio-left trials, whereas left-adapted subjects showed stronger effects for audio-right trials. The audio-center condition generally showed weaker discrimination.

3. EEG acquisition and analytical formalism

EEG was recorded with 60 Ag/AgCl electrodes arranged according to the 10–10 system, referenced to the average of left and right earlobes. Eye movements were monitored with an electrode beneath the right eye for blinks and vertical eye movement, and electrodes at the left and right outer canthi for horizontal eye movement. The sampling rate was 500 Hz. Offline filtering used a high cutoff: 40 Hz and a Butterworth filter, order 5. The provided text does not report additional preprocessing such as ICA, artifact rejection thresholds, re-referencing changes, baseline interval specifics, or epoch durations beyond what is stated (Saha, 16 Jan 2026).

Spectral decomposition was performed with Short-Time Fourier Transform (STFT) using MATLAB’s spectrogram function, with window length: 20 time points, overlap: 25%, and 40 linearly spaced frequency points. For each subject, condition, and channel, this yielded a three-dimensional array of frequency points $\times$ time points $\times$ trials. Each subject contributed 6 conditions, defined by pre-training versus post-training and auditory location left, center, or right.

Functional connectivity networks were constructed using Phase Locking Value (PLV) and the imaginary part of coherence (ImCoh / imaginary coherency). For every subject, condition, time window, and frequency point, a $60 \times 60$ adjacency matrix was computed among electrodes. The local clustering coefficient was defined as

$CC_i=\frac{L_i}{k_i(k_i-1)},$

where $0^\circ$0 is the degree of node $0^\circ$1 and $0^\circ$2 is the number of links among its neighbors. Modularity was defined as

$0^\circ$3

with $0^\circ$4, $0^\circ$5 denoting adjacency entries, and $0^\circ$6 equal to 1 when nodes $0^\circ$7 and $0^\circ$8 belong to the same community and 0 otherwise. The participation coefficient was defined conceptually as the ratio between intermodular edges and intramodular edges, and used as a measure of hubness or intermodular integration (Saha, 16 Jan 2026).

For PLV, the analytic signal was written as

$0^\circ$9

with Hilbert transform

$+10^\circ$0

and

$+10^\circ$1

The paper explicitly notes that PLV is sensitive to volume conduction, because a common source can induce spurious synchrony at nearby electrodes. By contrast, the use of imaginary coherence follows the argument of Nolte et al. (2004): under a linear mixing model with a common instantaneous source, coherency is purely real, so the imaginary part preferentially reflects delayed interactions and is less contaminated by common-source artifacts. This distinction is methodologically central because the more interpretable network findings were obtained with ImCoh, not PLV (Saha, 16 Jan 2026).

The thesis also includes a substantial review of Empirical Mode Decomposition (EMD) for non-stationary, non-linear EEG signals. The signal representation

$+10^\circ$2

was used, with instantaneous frequency

$+10^\circ$3

The sifting process was described by

$+10^\circ$4

with stopping based on

$+10^\circ$5

using a recommended tolerance between 0.2 and 0.3. In the classification analysis, baseline-removed raw EEG from each electrode and trial was decomposed into intrinsic mode functions, and the first IMF was used as classifier input. In this setting, EMD was therefore not a stand-alone interpretive model, but a feature representation intended to capture informative non-stationary oscillatory structure.

4. Network characterization of recalibration

Community detection was performed with the Louvain algorithm, initialized with each node as its own community, followed by iterative reassignment of nodes to neighboring communities when modularity improved, aggregation into supernodes, and repetition until modularity could no longer be improved. Because Louvain can yield different partitions across runs, the analysis repeatedly sampled the high-modularity plateau and averaged measures over multiple community assignments. To assess whether modularity was meaningful, the study constructed 25 Newman–Girvan null networks for each network by randomizing edges while preserving degree, weight, and strength distributions. All obtained networks were reported as significantly modular, with $+10^\circ$6 (Saha, 16 Jan 2026).

The most specific network findings concerned Imaginary Coherence. Nodal clustering coefficient was reported as significantly greater post-training than pre-training at $+10^\circ$7, especially in the 100–150 ms window, for right-adapted subjects in the audio-left condition and left-adapted subjects in the audio-right condition, in alpha, beta, and theta bands. For the audio-center condition, differences appeared mainly before stimulus presentation in alpha and beta and late after stimulus in theta. However, these effects did not survive Bonferroni correction or False Discovery Rate correction. For PLV networks, no significant pre/post differences in nodal clustering coefficient were found in any condition or band (Saha, 16 Jan 2026).

The analysis of modularity and number of modules yielded no significant differences. The thesis attributes this to the coarseness of these features and to high across-subject variance. In contrast, participation coefficients derived from ImCoh networks showed that centro-parietal electrodes were significantly greater post-training around the 100 ms time window in the alpha band for audio-left and audio-right conditions in right-adapted subjects and the audio-right condition in left-adapted subjects. These effects again did not survive Bonferroni correction (Saha, 16 Jan 2026).

The interpretation advanced in the thesis is that recalibration may involve not only altered local sensory coding but also changes in intermodular connectivity, with centro-parietal regions acting as hubs. Because the corrected significance is weak, the network-level conclusions are best treated as exploratory. Even so, the selective appearance of effects in ImCoh rather than PLV is methodologically consequential. It suggests that the informative changes are more consistent with delayed functional interactions than with spurious local synchrony induced by scalp field spread. A plausible implication is that recalibration changes the topology of task-relevant communication without requiring gross reorganization of coarse community structure.

5. Multivariate decoding and temporal localization

Multivariate classification was performed with Classwise Principal Component Analysis (CPCA), following Das and Nenadic (2009), and was motivated by the high-dimensional, small-sample character of EEG. Classification was carried out subject-wise to discriminate pre-training versus post-training EEG while holding physical sound location fixed. The classifier was thus intended to distinguish a sound from its post-adaptation “ventriloquized” version. The analysis used 10-fold cross-validation, the performance metric PC, and significance testing against chance with the Wilcoxon signed-rank test (Saha, 16 Jan 2026).

Using the full-trial raw EEG across all electrodes, decoding was above chance in all six condition-by-group combinations. For right-adapted subjects, mean PC was 0.5658 for audio-left, 0.5501 for audio-center, and 0.5650 for audio-right, all with $+10^\circ$8. For left-adapted subjects, mean PC was 0.5251 for audio-left with $+10^\circ$9, 0.5247 for audio-center with $15^\circ$0, and 0.5279 for audio-right with $15^\circ$1. The emphasized pattern was that audio-center decoding is weaker than the lateral conditions, and that the strongest decoding appears in the adaptation-consistent lateral contrast: right-adapted $15^\circ$2 audio-left strongest and left-adapted $15^\circ$3 audio-right strongest (Saha, 16 Jan 2026).

A parallel analysis using the first intrinsic mode function (IMF1) from each electrode’s baseline-removed trial preserved above-chance classification without radically improving it. For right-adapted subjects, mean PC was 0.5643 for audio-left, 0.5501 for audio-center, and 0.5630 for audio-right, all with $15^\circ$4. For left-adapted subjects, mean PC was 0.5351 for audio-left with $15^\circ$5, 0.5238 for audio-center with $15^\circ$6, and 0.5249 for audio-right with $15^\circ$7. This suggests that a substantial part of the information distinguishing pre- and post-adaptation trials is present in the fast, non-stationary oscillatory components isolated by EMD (Saha, 16 Jan 2026).

To address temporal decay, the study repeated classification using only the first 50% of post-training trials after each training session as “early trials.” For right-adapted subjects, mean PC was 0.5541 for audio-left, 0.5443 for audio-center, and 0.5556 for audio-right, each with $15^\circ$8. For left-adapted subjects, mean PC was 0.5459 for audio-left, 0.5233 for audio-center, and 0.5251 for audio-right, each with $15^\circ$9. The paper argues that because classification based only on early post-training trials was similar to classification using all post-training trials, most discriminatory information is concentrated early after exposure, implying that the after-effect decays with time or trials after training. This is an indirect argument; no fitted decay curve, time constant, regression slope, or explicit effect-size estimate is reported.

The strongest temporal localization result came from time-window-wise decoding. In this analysis, EEG was time-averaged within each window for each electrode, producing feature vectors whose dimensionality equaled the number of electrodes. For both left- and right-adapted subjects, classification was significantly above chance in each time window between 100 ms and 260 ms for audio-left and audio-right conditions, whereas audio-center was not significantly above chance. This 100–260 ms interval is presented as one of the strongest results in the thesis, indicating that EEG patterns distinguishing pre- from post-training auditory processing emerge in a relatively early post-stimulus interval (Saha, 16 Jan 2026).

6. Neural interpretation, theoretical placement, and limitations

Across methods, the strongest effects converge on early post-stimulus windows: network clustering changes at 100–150 ms, participation coefficient changes around 100 ms, and classification above chance from 100–260 ms. The thesis interprets this convergence as evidence that recalibration occurs early in the auditory processing pathway, rather than solely at late decisional or response stages (Saha, 16 Jan 2026). Because the study does not perform source localization, the anatomical locus cannot be pinpointed with high precision. The discussion nevertheless connects the findings to earlier literature implicating auditory cortex, especially planum temporale, in ventriloquism-related spatial processing. Within the scalp-level results themselves, the most specific network observation concerns centro-parietal electrodes as candidate hubs with increased participation.

Several limitations qualify the strength of the conclusions. Most importantly, many network effects were initially significant at $15^\circ$0 but did not survive Bonferroni correction, and the clustering effects also failed after False Discovery Rate correction. This places the network findings in an exploratory rather than definitive category. The thesis also does not report explicit behavioral effect-size measures such as localization bias magnitudes, psychometric shifts, confidence intervals, or direct behavioral-neural correlations. The interpretation of neural recalibration is therefore not tightly anchored to a quantitative behavioral estimate in the provided text (Saha, 16 Jan 2026).

The evidence for decay is also coarse. The argument derives from the similarity of classification using only early post-training trials to classification using all post-training trials. This supports the presence of decay, but does not quantify an exponential constant, a linear rate, or trial-by-trial behavioral-neural coupling. A plausible implication is that the adaptation signature is strongest shortly after exposure and weakens thereafter, but the thesis does not model that weakening directly.

Finally, the classification accuracies are statistically above chance yet modest, mostly around 0.52 to 0.57. This indicates that the neural distinction between pre- and post-adaptation auditory processing is real but subtle. Within those constraints, the work contributes a technically rich synthesis in which network theory, non-stationary signal decomposition, and multivariate decoding jointly support the view that ventriloquism after-effects reflect an early alteration of auditory spatial processing, with subsequent decay over post-exposure time or trials.