Frequency Test-Time Training (F-TTT)

• Frequency Test-Time Training (F-TTT) is a frequency-aware test-time adaptation strategy that adjusts model parameters using instance-specific spectral features for improved denoising and alignment under distribution shifts.
• It employs Fourier-domain modules—such as proxy spectral reconstruction in PET and band-stop filtering in EEG—to suppress global noise and recover high-frequency details.
• Empirical results show that F-TTT outperforms spatial-only methods, achieving notable gains in metrics like PSNR, AUROC, and Cohen’s kappa across different application domains.

Frequency Test-Time Training (F-TTT) denotes frequency-aware test-time adaptation mechanisms in "U-TTT: Towards Generalizable PET Image Denoising via Test-Time Training" (Yang et al., 9 Jun 2026) and "NeuroTTT: Bridging Pretraining-Downstream Task Misalignment in EEG Foundation Models via Test-Time Training" (Wang et al., 30 Sep 2025). In U-TTT, F-TTT is a Fourier-domain module embedded in a U-shaped PET denoiser, where a single-step proxy spectral reconstruction updates block-specific inner-loop weights during inference. In NeuroTTT, F-TTT is an inference-time procedure for EEG foundation models that adapts the backbone on unlabeled test samples using spectral self-supervision or, alternatively, prediction entropy minimization. Taken together, these formulations suggest a shared emphasis on per-instance spectral adaptation under distribution shift.

1. Scope, motivation, and problem setting

The two formulations arise from different failure modes. In PET denoising, the stated motivation is that residual noise is not purely a local spatial phenomenon but often exhibits structured, global patterns in the frequency domain, including periodic detector artifacts and broad-band noise; a spatial-only adaptation can correct local structural degradations but cannot selectively suppress these global noise spectra or recover attenuated high-frequency details. In EEG foundation models, the stated motivation is misalignment between pretraining objectives and downstream tasks, together with significant cross-subject distribution shifts; the spectral component is introduced to align latent representations to important spectral EEG features and to recalibrate them per test sample (Yang et al., 9 Jun 2026, Wang et al., 30 Sep 2025).

Dimension	U-TTT	NeuroTTT
Application domain	3D PET image denoising	EEG foundation models for BCI tasks
Primary motivation	Structured global frequency-domain noise and attenuated high-frequency detail	Pretraining-downstream task misalignment and cross-subject distribution shifts
Frequency adaptation mechanism	FFT-domain self-reconstruction inside each block	Band-stop spectral self-supervision and inference-time adaptation

This comparison makes clear that F-TTT is not tied to a single downstream modality. The common element is the decision to expose a model to a self-supervised frequency-oriented objective at test time, so that parameters or calibration statistics can be adjusted to the specific spectral profile of the current input.

2. Fourier-domain F-TTT in U-TTT

In U-TTT, every level of the U-shaped encoder-decoder interleaves an S-TTT block with an F-TTT block. The S-TTT layer captures and corrects spatial structural degradations, while the F-TTT layer suppresses global noise spectra and restores delicate high-frequency details. Each F-TTT block replaces the self-attention module in a standard Transformer-style encoder/decoder block (Yang et al., 9 Jun 2026).

Its forward pass on an input feature map $$F_{\mathrm{in}} \in \mathbb{R}^{D\times H\times W\times C}$$ has three stages. First, a $1\times1\times1$ convolution lifts $F_{\mathrm{in}}$ to a richer representation, and a 3D Fast Fourier Transform maps it to the complex spectral domain:

$$S=\mathcal{F}\bigl(\mathrm{Conv}_{1\times1\times1}(F_{\mathrm{in}})\bigr)\in\mathbb{C}^{D\times H\times W\times C}.$$

The spectrum is then split along the channel dimension into three equal chunks,

$$[F_1,F_2,F_3]=\mathrm{ChannelSplit}(S),\qquad F_i\in\mathbb{C}^{D\times H\times W\times C/3},$$

where $F_1$ and $F_2$ form the input and target for a spectral reconstruction proxy task, and $F_3$ is the test feature to be refined.

Second, the inner Frequency-Reconstruction Model (FRM) operates purely token-wise, since every frequency “pixel” already carries global context. Its forward map is

$$\widehat F_1=\mathrm{FRM}(W;F_1)=\bigl(\mathrm{FC}(F_1)\bigr)\odot \mathrm{SiLU}\!\bigl(\mathrm{FC}(F_1)\bigr),$$

where $\mathrm{FC}$ denotes a learned linear mapping, $1\times1\times1$0 is the sigmoid-weighted linear unit, and $1\times1\times1$1 is element-wise multiplication. Reconstruction quality is measured by a dot-product loss,

$1\times1\times1$2

with $1\times1\times1$3 the complex conjugate.

Third, test-time adaptation applies one gradient step to the FRM weights:

$1\times1\times1$4

where $1\times1\times1$5 is a learnable inner-loop learning rate. The adapted FRM is then applied to $1\times1\times1$6, followed by inverse FFT and a final $1\times1\times1$7 convolution:

$1\times1\times1$8

A central architectural property is that the full inner loop—forward pass, proxy loss, and update—is unrolled in the global graph. During training, the outer denoising objective backpropagates through this inner loop, so the model meta-learns to align $1\times1\times1$9 with the ultimate PET denoising loss.

3. Optimization behavior and empirical impact in PET denoising

The U-TTT formulation updates only the inner-loop FRM weights $F_{\mathrm{in}}$0, with one distinct copy per F-TTT block, and uses a single gradient step per block per test instance. The inner-loop learning rate $F_{\mathrm{in}}$1 is learned during meta-training and is described as typically on the order of $F_{\mathrm{in}}$2–$F_{\mathrm{in}}$3. The stated effect is that backpropagation against $F_{\mathrm{in}}$4 teaches the FRM to attenuate frequency bands dominated by noise and to amplify or faithfully pass high-frequency bands carrying true structural detail (Yang et al., 9 Jun 2026).

The ablation study isolates the contribution of the frequency component. A baseline U-Net without any TTT layers obtains PSNR values of 47.63 dB on Base, 45.68 dB on OOD-DRF, and 41.89 dB on OOD-Scanner. Adding S-TTT only yields 48.45 dB, 46.07 dB, and 42.43 dB. Adding F-TTT only yields 48.72 dB, 46.55 dB, and 42.78 dB. Combining S-TTT and F-TTT yields 48.91 dB, 46.86 dB, and 43.10 dB. These figures place the frequency-only variant ahead of the spatial-only variant in all three reported conditions.

The full comparison further attributes quantitative gains to U-TTT. On in-distribution average dose reduction factors, U-TTT exceeds the prior best, VQPET, by $F_{\mathrm{in}}$5 dB PSNR and $F_{\mathrm{in}}$6 SSIM. Under OOD dose shift at DRF 4/10, it gains $F_{\mathrm{in}}$7 dB PSNR over VQPET, and under OOD scanner shift it gains $F_{\mathrm{in}}$8 dB. Qualitatively, F-TTT is reported to reduce “global haze” and restore crisp boundaries on small lesions that spatial-only methods tend to over-smooth. The paper summarizes the layer as lightweight and fully differentiable, with on-the-fly adaptation of frequency-processing kernels and substantially enhanced robustness to unseen dose levels and scanner types.

4. Frequency Test-Time Training in NeuroTTT

NeuroTTT organizes F-TTT in a two-stage pipeline. Stage I performs domain-tuned self-supervision during fine-tuning of an EEG foundation backbone $F_{\mathrm{in}}$9, such as CBraMod or LaBraM, with a main classification head $$S=\mathcal{F}\bigl(\mathrm{Conv}_{1\times1\times1}(F_{\mathrm{in}})\bigr)\in\mathbb{C}^{D\times H\times W\times C}.$$0 and three self-supervised heads: a spectral head $$S=\mathcal{F}\bigl(\mathrm{Conv}_{1\times1\times1}(F_{\mathrm{in}})\bigr)\in\mathbb{C}^{D\times H\times W\times C}.$$1, a spatial head $$S=\mathcal{F}\bigl(\mathrm{Conv}_{1\times1\times1}(F_{\mathrm{in}})\bigr)\in\mathbb{C}^{D\times H\times W\times C}.$$2, and a temporal head $$S=\mathcal{F}\bigl(\mathrm{Conv}_{1\times1\times1}(F_{\mathrm{in}})\bigr)\in\mathbb{C}^{D\times H\times W\times C}.$$3. The joint objective is

$$S=\mathcal{F}\bigl(\mathrm{Conv}_{1\times1\times1}(F_{\mathrm{in}})\bigr)\in\mathbb{C}^{D\times H\times W\times C}.$$4

where $$S=\mathcal{F}\bigl(\mathrm{Conv}_{1\times1\times1}(F_{\mathrm{in}})\bigr)\in\mathbb{C}^{D\times H\times W\times C}.$$5 are self-supervised views and $$S=\mathcal{F}\bigl(\mathrm{Conv}_{1\times1\times1}(F_{\mathrm{in}})\bigr)\in\mathbb{C}^{D\times H\times W\times C}.$$6 are pseudo-labels (Wang et al., 30 Sep 2025).

The spectral self-supervised task is Stopped-Band Prediction. For raw EEG $$S=\mathcal{F}\bigl(\mathrm{Conv}_{1\times1\times1}(F_{\mathrm{in}})\bigr)\in\mathbb{C}^{D\times H\times W\times C}.$$7, the transformed view is $$S=\mathcal{F}\bigl(\mathrm{Conv}_{1\times1\times1}(F_{\mathrm{in}})\bigr)\in\mathbb{C}^{D\times H\times W\times C}.$$8 and the pseudo-label is the band index $$S=\mathcal{F}\bigl(\mathrm{Conv}_{1\times1\times1}(F_{\mathrm{in}})\bigr)\in\mathbb{C}^{D\times H\times W\times C}.$$9. The paper specifies that each component loss is a cross-entropy, or permutation-cross-entropy for the temporal jigsaw task.

Stage II defines F-TTT at inference. Two lightweight adaptation options are available. In self-supervised TTT, the spectral SSL view $$[F_1,F_2,F_3]=\mathrm{ChannelSplit}(S),\qquad F_i\in\mathbb{C}^{D\times H\times W\times C/3},$$0 is created by band-stop filtering one random EEG band in the unlabeled test sample $$[F_1,F_2,F_3]=\mathrm{ChannelSplit}(S),\qquad F_i\in\mathbb{C}^{D\times H\times W\times C/3},$$1, the spectral loss is

$$[F_1,F_2,F_3]=\mathrm{ChannelSplit}(S),\qquad F_i\in\mathbb{C}^{D\times H\times W\times C/3},$$2

and the model takes one gradient descent step on the full parameters:

$$[F_1,F_2,F_3]=\mathrm{ChannelSplit}(S),\qquad F_i\in\mathbb{C}^{D\times H\times W\times C/3},$$3

The adapted prediction is then $$[F_1,F_2,F_3]=\mathrm{ChannelSplit}(S),\qquad F_i\in\mathbb{C}^{D\times H\times W\times C/3},$$4.

The alternative option is Tent. Here the softmax output is $$[F_1,F_2,F_3]=\mathrm{ChannelSplit}(S),\qquad F_i\in\mathbb{C}^{D\times H\times W\times C/3},$$5, the entropy loss is

$$[F_1,F_2,F_3]=\mathrm{ChannelSplit}(S),\qquad F_i\in\mathbb{C}^{D\times H\times W\times C/3},$$6

and only the BatchNorm parameters $$[F_1,F_2,F_3]=\mathrm{ChannelSplit}(S),\qquad F_i\in\mathbb{C}^{D\times H\times W\times C/3},$$7 are updated:

$$[F_1,F_2,F_3]=\mathrm{ChannelSplit}(S),\qquad F_i\in\mathbb{C}^{D\times H\times W\times C/3},$$8

5. Practical regime and empirical gains in EEG

NeuroTTT gives explicit hyperparameters for both stages. In Stage I, optimization uses Adam with learning rate $$[F_1,F_2,F_3]=\mathrm{ChannelSplit}(S),\qquad F_i\in\mathbb{C}^{D\times H\times W\times C/3},$$9 and weight decay $F_1$0. The SSL weights are chosen so that each SSL loss is approximately the same order of magnitude as $F_1$1: for Imagined Speech, $F_1$2; for Mental Stress, $F_1$3; for Motor Imagery, $F_1$4. Batch size is 32–64, training continues until the validation metric plateaus at roughly 20–150 epochs, and dropout $F_1$5 is applied on SSL heads to prevent overfitting to transformations. In Stage II, SSL-TTT uses one step per sample, batch size $F_1$6, and $F_1$7; Tent uses one update per batch with $F_1$8 and updates only BN affine parameters. The method warm-starts $F_1$9 at the end of fine-tuning each subject or session, resets after each subject if data are IID, carries parameters forward if streaming, and uses gradient clipping $F_2$0 to avoid catastrophic drifts on noisy trials (Wang et al., 30 Sep 2025).

The ablation results quantify the contribution of the frequency component. On BCI Competition 2020-III for Imagined Speech with 5 classes, Full supervised FT reaches balanced accuracy 0.473; adding $F_2$1 only yields 0.503; adding $F_2$2 only yields 0.492; adding $F_2$3 yields 0.517; adding all SSL tasks yields 0.533; adding F-TTT with SSL-TTT yields 0.589; and adding F-TTT with Tent yields 0.590. On Mental Stress Detection, AUROC rises from 0.779 for Full supervised FT to 0.805 with $F_2$4 only, 0.791 with $F_2$5 only, 0.814 with both SSL tasks, 0.825 with F-TTT (SSL-TTT), and 0.826 with F-TTT (Tent). On BCIC-IV-2a for Motor Imagery, Cohen’s $F_2$6 rises from 0.337 to 0.361 with $F_2$7 only, 0.350 with $F_2$8 only, 0.379 with both SSL tasks, 0.391 with F-TTT (SSL-TTT), and 0.423 with F-TTT (Tent).

The paper’s key insights are explicit. Spectral SSL alone yields a consistent $F_2$9–$F_3$0 point gain across all tasks, indicating that frequency-aware pretext learning re-aligns the backbone to the bands most predictive of each BCI task. Adding spatial or temporal tasks further boosts representation quality. Frequency TTT on a single sample contributes an additional $F_3$1–$F_3$2 points, which the paper interprets as evidence that adapting spectral sensitivity to each trial’s specific noise and amplitude profile effectively mitigates subject or session shifts. Tent yields similar or slightly better improvements, especially in cross-subject motor imagery, by recalibrating feature statistics.

6. Conceptual boundaries, recurring patterns, and points of clarification

A recurrent source of ambiguity is that F-TTT is not presented as a single canonical algorithm across these works. In U-TTT, F-TTT is an architectural layer embedded throughout a U-shaped PET denoiser; in NeuroTTT, F-TTT is an inference-time adaptation stage attached to a pretrained EEG foundation model (Yang et al., 9 Jun 2026, Wang et al., 30 Sep 2025). The shared label therefore denotes a frequency-oriented test-time adaptation strategy rather than a fixed implementation.

Another point of clarification is that F-TTT is not equivalent to simple frequency preprocessing. In U-TTT, FFT is only one component of a block that also includes channel splitting, a token-wise FRM, a complex-domain dot-product proxy loss, a learnable inner-loop learning rate, and inverse FFT reconstruction. In NeuroTTT, the spectral component is operationalized through band-stop filtering and pseudo-label prediction, with either a one-step full-parameter update or entropy minimization over BatchNorm parameters. These are adaptation procedures, not merely spectral feature extraction.

The role of what is updated also differs. U-TTT updates only the inner-loop FRM weights $F_3$3 for each F-TTT block. NeuroTTT’s SSL-TTT updates the full parameters $F_3$4, while Tent updates only $F_3$5. This distinction matters because the two papers target different forms of shift: unseen dose levels and unseen scanners in PET, versus subject and session variability, together with pretraining-downstream misalignment, in EEG.

A plausible implication is that F-TTT, as currently formulated, is best understood as a design principle for aligning instance-specific spectral statistics with downstream objectives under distribution shift. The available formulations are united by the use of lightweight self-supervised or calibration objectives at inference, but they differ in whether the frequency adaptation is implemented as an internal block-level reconstruction mechanism or as a sample-level update rule.