WAFL-ViT: Federated & Wavelet Vision Transformer

• The paper presents a novel WAFL-ViT approach that fuses a frozen Vision Transformer backbone with decentralized federated learning by exchanging only the classifier head among edge devices.
• WAFL-ViT encompasses multiple interpretations—including Wireless Ad Hoc Federated Learning, Wavelet Vision Transformer, and Multiscale Wavelet Attention—each addressing unique design challenges.
• Empirical results on mission-oriented datasets demonstrate superior accuracy and consistency over self-training baselines, while wavelet-based modules enhance token mixing and preserve fine image details.

Searching arXiv for the papers and terminology to ground the article in current records. WAFL with Vision Transformer (WAFL-ViT) appears in the provided literature in three distinct contexts rather than as a single uniformly defined model. In one usage, it denotes Wireless Ad Hoc Federated Learning (WAFL) with a Vision Transformer (ViT) backbone for mission-oriented image recognition on decentralized edge devices (Taniguchi et al., 5 Dec 2025). In a second usage, the label is treated as a naming variant or misnomer for Wavelet Vision Transformer (Wave-ViT), which replaces non-invertible pooling-based key/value down-sampling in multi-scale ViTs with an invertible discrete wavelet transform and uses inverse DWT to enrich attention outputs (Yao et al., 2022). In a third usage, the term is mapped to a ViT in which self-attention is replaced by Multiscale Wavelet Attention (MWA) derived from wavelet neural operators (Nekoozadeh et al., 2023). The shared theme across these usages is the combination of transformer-based visual representations with either decentralized collaborative learning or wavelet-domain multiscale processing.

1. Terminological scope and disambiguation

The expression “WAFL-ViT” is not a single canonical designation in the cited material. The following usages are explicitly present.

Usage	Core meaning	Primary source
WAFL-ViT	Wireless Ad Hoc Federated Learning with a ViT backbone; only the classifier MLP head is fine-tuned and exchanged among neighbors	(Taniguchi et al., 5 Dec 2025)
“WAFL-ViT” as a naming variant or misnomer	Wavelet Vision Transformer (Wave-ViT), which formulates invertible down-sampling with wavelet transforms and self-attention learning in a unified way	(Yao et al., 2022)
WAFL mapped to MWA-ViT	Multiscale Wavelet Attention replaces self-attention in ViT blocks through wavelet-domain filtering and IDWT reconstruction	(Nekoozadeh et al., 2023)

For the wavelet-based interpretation associated with Wave-ViT, the source explicitly states that there is no separate prior called “WAFL” in that context, and that “WAFL with Vision Transformer (WAFL-ViT)” appears to be a naming variant or misnomer for Wavelet Vision Transformer (Wave-ViT) (Yao et al., 2022). For the wavelet-neural-operator interpretation, the source explicitly maps “WAFL” to the proposed Multiscale Wavelet Attention (MWA) module that replaces self-attention (Nekoozadeh et al., 2023). By contrast, in the mission-oriented IoT paper, WAFL unambiguously means Wireless Ad Hoc Federated Learning, a fully decentralized, device-to-device collaborative training framework for edge devices in an ad hoc wireless network (Taniguchi et al., 5 Dec 2025).

This terminological overlap is a recurring source of confusion. A careful reading therefore requires distinguishing between WAFL as a decentralized learning protocol and WAFL as a wavelet-based token-mixing or attention replacement shorthand.

2. WAFL-ViT as Wireless Ad Hoc Federated Learning with a Vision Transformer

In the mission-oriented IoT setting, WAFL is defined as a fully decentralized, device-to-device collaborative training framework for edge devices in an ad hoc wireless network (Taniguchi et al., 5 Dec 2025). It was proposed to address privacy and single-point-of-failure concerns inherent to centralized federated learning. Edge devices train locally on their own data and exchange model information only with their wireless neighbors, thereby forming a mission-oriented sensor system. The deployment emphasized in the source is smart-campus building image recognition, in which sensors distributed across a campus collaboratively learn a joint classifier without relying on a central server.

WAFL-ViT integrates a Vision Transformer (ViT) into that paradigm. The key design choice is explicit: freeze the ViT backbone on each device and only fine-tune and exchange the parameters of the classifier MLP head among neighbors (Taniguchi et al., 5 Dec 2025). The source states that this design limits overfitting to the local (often limited) data on each device and reduces communication overhead, because only the small head is shared, not the entire ViT. It also states, verbatim, that “To avoid overfitting to the local images, all parameters of each ViT in each device are not fine-tuned”.

The local supervised objective is standard multi-class cross-entropy. For device $n$ with local dataset $D_n=\{(x_i,y_i)\}$, the loss is given as

$$\ell_{\mathrm{CE}}(y,p)=-\sum_{c=1}^{C}\mathbf{1}[y=c]\log p_c,$$

with $p=\mathrm{softmax}(\mathrm{logits})$, and the local objective is

$$L_n(W^n)=\frac{1}{|D_n|}\sum_{(x,y)\in D_n}\ell_{\mathrm{CE}}\!\left(y,f_{\text{head}}\!\left(f_{\text{ViT}}(x;\theta_{\text{ViT}});W^n\right)\right),$$

where $\theta_{\text{ViT}}$ are frozen ViT backbone parameters and $W^n$ are the MLP head parameters on device $n$ (Taniguchi et al., 5 Dec 2025).

The ViT backbone described in the same source is ViT-B/16. Patch embedding is defined in the standard form by splitting an input image $X\in\mathbb{R}^{H\times W\times 3}$ into non-overlapping $p\times p$ patches, with $D_n=\{(x_i,y_i)\}$0, and projecting them linearly:

$D_n=\{(x_i,y_i)\}$1

where $D_n=\{(x_i,y_i)\}$2 is the learned patch projection and $D_n=\{(x_i,y_i)\}$3 are learned positional encodings. A class token $D_n=\{(x_i,y_i)\}$4 may be prepended, as in standard ViT. Self-attention and the transformer encoder block are written in standard form:

$D_n=\{(x_i,y_i)\}$5

$D_n=\{(x_i,y_i)\}$6

The exchanged classifier head consumes either the $D_n=\{(x_i,y_i)\}$7 token representation or pooled patch features to produce logits over $D_n=\{(x_i,y_i)\}$8 classes (Taniguchi et al., 5 Dec 2025).

3. Decentralized optimization, communication, and mission-oriented data

WAFL-ViT performs decentralized neighbor averaging of the MLP head parameters (Taniguchi et al., 5 Dec 2025). Let $D_n=\{(x_i,y_i)\}$9 index devices, $$\ell_{\mathrm{CE}}(y,p)=-\sum_{c=1}^{C}\mathbf{1}[y=c]\log p_c,$$0 be the set of wireless neighbors of device $$\ell_{\mathrm{CE}}(y,p)=-\sum_{c=1}^{C}\mathbf{1}[y=c]\log p_c,$$1, $$\ell_{\mathrm{CE}}(y,p)=-\sum_{c=1}^{C}\mathbf{1}[y=c]\log p_c,$$2 the MLP head parameters at device $$\ell_{\mathrm{CE}}(y,p)=-\sum_{c=1}^{C}\mathbf{1}[y=c]\log p_c,$$3, and $$\ell_{\mathrm{CE}}(y,p)=-\sum_{c=1}^{C}\mathbf{1}[y=c]\log p_c,$$4 a mixing hyperparameter. The parameter-exchange update is

$$\ell_{\mathrm{CE}}(y,p)=-\sum_{c=1}^{C}\mathbf{1}[y=c]\log p_c,$$5

The round structure is also explicit. In each round, a device $$\ell_{\mathrm{CE}}(y,p)=-\sum_{c=1}^{C}\mathbf{1}[y=c]\log p_c,$$6 first fine-tunes its local MLP head $$\ell_{\mathrm{CE}}(y,p)=-\sum_{c=1}^{C}\mathbf{1}[y=c]\log p_c,$$7 on local images while the ViT backbone remains frozen, and then exchanges $$\ell_{\mathrm{CE}}(y,p)=-\sum_{c=1}^{C}\mathbf{1}[y=c]\log p_c,$$8 with neighbors and updates $$\ell_{\mathrm{CE}}(y,p)=-\sum_{c=1}^{C}\mathbf{1}[y=c]\log p_c,$$9 via the above rule. The paper further states that there is no global server and no FedAvg; instead, the algorithm is a decentralized neighbor averaging (gossip-like) rule specialized to the MLP head (Taniguchi et al., 5 Dec 2025).

Connectivity is dynamic because devices follow random waypoint mobility (RWP), so $p=\mathrm{softmax}(\mathrm{logits})$0 changes over time. The simulation assumes 10 distributed devices across the campus. Images are uniformly distributed across devices (IID), and the source explicitly notes that non-IID heterogeneity is not considered in this study. Device hardware, bandwidth, latency, payload sizes per round, compression or quantization, and energy consumption are not specified (Taniguchi et al., 5 Dec 2025).

The mission-oriented data regime is central to this formulation. The paper cites the UTokyo Building Recognition Dataset (UTBR) as the original mission-oriented dataset used to validate WAFL-ViT; UTBR comprises 10 labeled buildings for smart-campus image recognition at the University of Tokyo, although the source does not provide per-class counts, splits, or resolutions there (Taniguchi et al., 5 Dec 2025). The new dataset introduced in the paper is the Chulalongkorn University Building Recognition Dataset (CUBR), tailored to smart-campus vision recognition in a WAFL setting. CUBR contains 32 distinct buildings (labels 0–31) and 4005 images across 32 classes, with totals of 3194 training images and 811 validation images. Images were captured over a month using 4 different devices and include diverse distances, viewing angles, illumination conditions, partial occlusions, and cluttered backgrounds (Taniguchi et al., 5 Dec 2025).

This mission-oriented framing is presented as the rationale for dataset specialization. The source states that WAFL is built for targeted missions where devices collect local data tied to the deployment environment, and that CUBR extends this paradigm to Chulalongkorn University so that the training data reflect the actual buildings and conditions encountered by an IoT sensor network on that campus (Taniguchi et al., 5 Dec 2025).

4. Empirical behavior of decentralized WAFL-ViT

The reported evaluation protocol computes accuracy at 10 nodes using test data, with mean and standard deviation calculated over 32 labels × 10 nodes = 320 accuracies (Taniguchi et al., 5 Dec 2025). Under the IID case on CUBR, the quantitative results are as follows.

Model	Accuracy	Standard deviation
WAFL-ViT	0.861	0.007
WAFL-ResNet	0.833	0.022
WAFL-VGG	0.732	0.008
WAFL-MobileNet	0.791	0.015
SELF-ViT	0.619	0.017
SELF-ResNet	0.596	0.025
SELF-VGG	0.498	0.023
SELF-MobileNet	0.561	0.018

The source states that WAFL-ViT achieves the best accuracy and consistency (lowest std) among all configurations, and that all WAFL models outperform their corresponding self-training baselines (Taniguchi et al., 5 Dec 2025). A confusion matrix is shown for device 6 at epoch 2000, and the accompanying analysis indicates difficulty distinguishing between label 10 and label 11.

The training budgets are asymmetric across scenarios. The paper reports “up to 500 epochs for self-training and up to 1500 epochs for collaborative training” for WAFL models, and “up to 2000 epochs for self-training” for SELF models. It does not specify per-round local steps or the mapping from epochs to communication rounds. It also does not include training curves, convergence statistics, or ablations such as the number of devices, communication rounds, ViT size, patch size, or learning rate (Taniguchi et al., 5 Dec 2025).

A plausible implication is that the numerical superiority of WAFL-ViT over self-training on CUBR is well supported within the reported IID simulation, whereas its behavior under non-IID partitions, heterogeneous devices, or explicit wireless constraints remains outside the scope of the reported experiments.

5. Wave-ViT: wavelet-based invertible down-sampling in multi-scale vision transformers

In the wavelet-transformer literature, the closely related model is Wave-ViT, introduced as Wavelet Vision Transformer (Yao et al., 2022). Its motivation is the quadratic cost of self-attention with respect to the input patch number. Multi-scale ViTs often reduce attention cost by down-sampling operations (e.g., average pooling) over keys/values, but the source argues that such over-aggressive down-sampling design is not invertible and inevitably causes information dropping especially for high-frequency components in objects (e.g., texture details).

Wave-ViT addresses that issue by replacing pooling-based $p=\mathrm{softmax}(\mathrm{logits})$1 down-sampling with an invertible discrete wavelet transform (DWT) and using inverse DWT (IDWT) to strengthen attention outputs by aggregating local contexts with enlarged receptive field (Yao et al., 2022). Standard scaled dot-product attention is written as

$p=\mathrm{softmax}(\mathrm{logits})$2

$p=\mathrm{softmax}(\mathrm{logits})$3

with complexity $p=\mathrm{softmax}(\mathrm{logits})$4. If $p=\mathrm{softmax}(\mathrm{logits})$5 and $p=\mathrm{softmax}(\mathrm{logits})$6 are computed on a reduced set of tokens $p=\mathrm{softmax}(\mathrm{logits})$7, the complexity becomes $p=\mathrm{softmax}(\mathrm{logits})$8. With DWT-based down-sampling at factor 2 per spatial dimension, $p=\mathrm{softmax}(\mathrm{logits})$9 (Yao et al., 2022).

The transform uses the Haar wavelet with filters

$$L_n(W^n)=\frac{1}{|D_n|}\sum_{(x,y)\in D_n}\ell_{\mathrm{CE}}\!\left(y,f_{\text{head}}\!\left(f_{\text{ViT}}(x;\theta_{\text{ViT}});W^n\right)\right),$$0

A single-level 2D DWT yields four sub-bands at half resolution in each spatial dimension:

$$L_n(W^n)=\frac{1}{|D_n|}\sum_{(x,y)\in D_n}\ell_{\mathrm{CE}}\!\left(y,f_{\text{head}}\!\left(f_{\text{ViT}}(x;\theta_{\text{ViT}});W^n\right)\right),$$1

$$L_n(W^n)=\frac{1}{|D_n|}\sum_{(x,y)\in D_n}\ell_{\mathrm{CE}}\!\left(y,f_{\text{head}}\!\left(f_{\text{ViT}}(x;\theta_{\text{ViT}});W^n\right)\right),$$2

These correspond to LL (coarse structure) and LH/HL/HH (horizontal, vertical, diagonal high-frequency textures). For orthonormal Haar, perfect reconstruction holds if filters form a quadrature mirror pair (QMF), and boundary handling uses symmetric padding in practice to avoid artifacts (Yao et al., 2022).

Wave-ViT uses a four-stage hierarchical ViT backbone (input 224×224) with outputs $$L_n(W^n)=\frac{1}{|D_n|}\sum_{(x,y)\in D_n}\ell_{\mathrm{CE}}\!\left(y,f_{\text{head}}\!\left(f_{\text{ViT}}(x;\theta_{\text{ViT}});W^n\right)\right),$$3, $$L_n(W^n)=\frac{1}{|D_n|}\sum_{(x,y)\in D_n}\ell_{\mathrm{CE}}\!\left(y,f_{\text{head}}\!\left(f_{\text{ViT}}(x;\theta_{\text{ViT}});W^n\right)\right),$$4, $$L_n(W^n)=\frac{1}{|D_n|}\sum_{(x,y)\in D_n}\ell_{\mathrm{CE}}\!\left(y,f_{\text{head}}\!\left(f_{\text{ViT}}(x;\theta_{\text{ViT}});W^n\right)\right),$$5, and $$L_n(W^n)=\frac{1}{|D_n|}\sum_{(x,y)\in D_n}\ell_{\mathrm{CE}}\!\left(y,f_{\text{head}}\!\left(f_{\text{ViT}}(x;\theta_{\text{ViT}});W^n\right)\right),$$6. Within each Wavelets block, the incoming feature $$L_n(W^n)=\frac{1}{|D_n|}\sum_{(x,y)\in D_n}\ell_{\mathrm{CE}}\!\left(y,f_{\text{head}}\!\left(f_{\text{ViT}}(x;\theta_{\text{ViT}});W^n\right)\right),$$7 is first projected to $$L_n(W^n)=\frac{1}{|D_n|}\sum_{(x,y)\in D_n}\ell_{\mathrm{CE}}\!\left(y,f_{\text{head}}\!\left(f_{\text{ViT}}(x;\theta_{\text{ViT}});W^n\right)\right),$$8, then passed through a single-level 2D DWT to produce $$L_n(W^n)=\frac{1}{|D_n|}\sum_{(x,y)\in D_n}\ell_{\mathrm{CE}}\!\left(y,f_{\text{head}}\!\left(f_{\text{ViT}}(x;\theta_{\text{ViT}});W^n\right)\right),$$9, concatenated along channels into $\theta_{\text{ViT}}$0, and processed by a $\theta_{\text{ViT}}$1 convolution to impose locality. Queries are obtained from the original-resolution $\theta_{\text{ViT}}$2, while down-sampled keys and values are obtained from the wavelet-domain representation. The final block output concatenates the multi-head attention outputs with the IDWT reconstruction $\theta_{\text{ViT}}$3 and projects them back, thereby combining global attention with enhanced local context (Yao et al., 2022).

The source reports the following ImageNet-1K results: Wave-ViT-S: 82.7% top-1, 96.2% top-5, 19.8M params, 4.3 GFLOPs; Wave-ViT-S⋆: 83.9% top-1, 96.6% top-5, 22.7M params, 4.7 GFLOPs; Wave-ViT-B⋆: 84.8% top-1, 97.1% top-5, 33.5M params, 7.2 GFLOPs; Wave-ViT-L⋆: 85.5% top-1, 97.3% top-5, 57.5M params, 14.8 GFLOPs (Yao et al., 2022). On COCO with RetinaNet $\theta_{\text{ViT}}$4, Wave-ViT-S AP=45.8 and Wave-ViT-B AP=47.2; on Mask R-CNN $\theta_{\text{ViT}}$5, Wave-ViT-S $\theta_{\text{ViT}}$6, $\theta_{\text{ViT}}$7 and Wave-ViT-B $\theta_{\text{ViT}}$8, $\theta_{\text{ViT}}$9. On ADE20K with UPerNet, Wave-ViT-S $W^n$0 and Wave-ViT-B $W^n$1 (Yao et al., 2022).

The ablation on down-sampling choices is particularly central to the Wave-ViT interpretation. For the small model, the source reports average pooling K/V: 3.9 GFLOPs, 82.0% top-1; pooling kernels (conv): 4.0 GFLOPs, 82.0% top-1; DWT only (invertible K/V down-sampling): 4.3 GFLOPs, 82.5% top-1; and DWT + Conv + IDWT (full Wavelets block): 4.3 GFLOPs, 82.7% top-1 (Yao et al., 2022). The source further states that preserving LH/HL/HH sub-bands maintains edges and fine textures crucial for detection and segmentation.

6. MWA-ViT: multiscale wavelet attention as a replacement for self-attention

A second wavelet-based interpretation maps WAFL-ViT to the architecture described in “Multiscale Attention via Wavelet Neural Operators for Vision Transformers” (Nekoozadeh et al., 2023). In that usage, WAFL corresponds to the proposed Multiscale Wavelet Attention (MWA) module, which replaces self-attention in ViT blocks.

The motivation is that standard self-attention has quadratic complexity in sequence length, while AFNO and GFN provide efficient global filtering but are less effective for small and moderate scale structures that commonly appear in natural images (Nekoozadeh et al., 2023). MWA instead uses wavelet-domain multiscale filtering. For a 2D token-grid $W^n$2, forward DWT produces approximation and detail coefficients, and the learned operator is written as

$W^n$3

The default configuration uses Haar wavelet and $W^n$4 decomposition level. In practical block form, the input token sequence is reshaped to a grid, decomposed into LL, LH, HL, HH, each sub-band is filtered by grouped 2D/3D convolutions with kernel sizes $W^n$5 and $W^n$6 followed by GeLU, and IDWT reconstructs the spatial feature map. The module also uses weighted skip connections ($W^n$7 and $W^n$8 conv + GeLU) to facilitate identity mapping and high-frequency preservation (Nekoozadeh et al., 2023).

The resulting transformer block is LayerNorm → MWA → Residual Add → MLP (FFN) → Residual Add, with positional encodings unchanged. The source gives the complexity comparison in asymptotic form: SA: $W^n$9, AFNO: $n$0, and MWA: $n$1 in sequence length, with memory $n$2 and no $n$3 attention matrix (Nekoozadeh et al., 2023).

The reported experiments use CIFAR-10/100 and Tiny-ImageNet, with ViT-XS/4, ViT-S/4, and ViT-B/4, patch size $n$4, Adam, cross-entropy loss, 300 epochs, LR $n$5, 5-epoch linear warm-up, cosine decay to $n$6, gradient clipping $n$7, and weight decay 0.05 (Nekoozadeh et al., 2023). Representative results include:

• On CIFAR-10, ViT-B/4 with MWA achieves Top-1 96.1, compared with 95.2 for AFNO and 95.3 for GFN.
• On CIFAR-10, ViT-S/4 with MWA achieves Top-1 95.3, compared with 93.7 for AFNO and 94.4 for GFN.
• On Tiny-ImageNet, ViT-S/4 with MWA achieves Top-1 81.40, compared with 79.98 for AFNO and 80.32 for GFN (Nekoozadeh et al., 2023).

The ablations identify the practical defaults. For the number of decomposition levels, $n$8 is fastest and competitive, although higher $n$9 can modestly increase accuracy at significant training-time cost. For wavelet type on CIFAR-100 with ViT-XS/4, the source reports Haar: Top-1 71.6%, Train $X\in\mathbb{R}^{H\times W\times 3}$0, db4: Top-1 71.3%, Train $X\in\mathbb{R}^{H\times W\times 3}$1, and db6: Top-1 71.6%, Train $X\in\mathbb{R}^{H\times W\times 3}$2. For DWT vs. DTCWT, it reports DWT: Top-1 71.6%, Train $X\in\mathbb{R}^{H\times W\times 3}$3 and DTCWT: Top-1 72.2%, Train $X\in\mathbb{R}^{H\times W\times 3}$4. For the skip branches, $X\in\mathbb{R}^{H\times W\times 3}$5 gives the best shown CIFAR-100 result at Top-1 71.6% (Nekoozadeh et al., 2023).

7. Limitations, misconceptions, and comparative interpretation

The most immediate misconception is terminological. In the cited material, WAFL-ViT does not identify one unique architecture. It may refer to a decentralized federated ViT, to Wave-ViT, or to MWA-ViT. Any technical discussion therefore requires the underlying paper or mechanism to be specified explicitly.

For Wireless Ad Hoc Federated Learning with ViT, the source lists several limitations: the data distribution is IID across devices; non-IID heterogeneity is not assessed; only the MLP head is fine-tuned and exchanged; and real-world wireless parameters (bandwidth, latency, energy) and hardware characteristics are not provided, with results obtained from simulation with RWP mobility (Taniguchi et al., 5 Dec 2025). The source also notes that optimizer, learning rates, batch sizes, augmentation, and the mixing parameter $X\in\mathbb{R}^{H\times W\times 3}$6 are not specified by the authors.

For Wave-ViT, the limitations stated in the source include small compute and implementation complexity from DWT/IDWT, the fact that the paper uses Haar and does not report alternative wavelet families ablations, and the need for correct sub-band channel grouping and padding because implementation errors degrade reconstruction (Yao et al., 2022). The source further suggests that benefits are strongest on texture-rich, dense prediction tasks, while gains may be smaller on extremely smooth or low-texture data.

For MWA-ViT, the reported limitations are sensitivity to wavelet choice, possible boundary effects and padding artifacts, reduced benefit in very low-resolution regimes, and substantial training-time growth at higher decomposition levels (Nekoozadeh et al., 2023). Haar is repeatedly identified as the practical speed-oriented default, while richer wavelets or DTCWT can improve detail modeling at higher cost.

Taken together, the three lines of work indicate two different but technically related patterns. One pattern uses ViTs as the backbone model class inside a decentralized learning protocol (Taniguchi et al., 5 Dec 2025). The other uses wavelets to modify the token-mixing mechanism or attention pathway inside the transformer itself (Yao et al., 2022, Nekoozadeh et al., 2023). A plausible implication is that the phrase “WAFL-ViT” should always be accompanied by an explicit expansion—Wireless Ad Hoc Federated Learning, Wavelet Vision Transformer, or Multiscale Wavelet Attention—to avoid conflating distributed optimization with wavelet-domain attention design.