FEWT: Frequency-Enhanced Wavelet-based Transformer

• FEWT is a framework that fuses multi-scale wavelet decomposition with Transformer-based imitation learning to enhance humanoid robot perception.
• It integrates Frequency-Enhanced Efficient Multi-Scale Attention (FE-EMA) and Time-Series Discrete Wavelet Transform (TS-DWT) to improve success rates by up to 30% in simulation and 6–12% in real-world tasks.
• By combining visual, temporal, and multimodal data within a CVAE policy, FEWT achieves robust and efficient action prediction for dynamic robotic manipulation.

Searching arXiv for FEWT and related frequency-enhanced wavelet-transformer papers. Frequency-Enhanced Wavelet-based Transformer (FEWT) is an imitation learning framework for enhanced perception in humanoid robots that fuses multi-scale time- and frequency-domain features within a Transformer-based policy architecture. It was introduced together with a hardware platform comprising a humanoid robot and an exoskeleton-style teleoperation cabin for intuitive remote manipulation and efficient collection of anthropomorphic action data. FEWT is defined by two primary modules—Frequency-Enhanced Efficient Multi-Scale Attention (FE-EMA) and Time-Series Discrete Wavelet Transform (TS-DWT)—and is reported to improve the success rate of the state-of-the-art algorithm Action Chunking with Transformers (ACT baseline) by up to 30% in simulation and by 6–12% in real-world settings (Huang et al., 14 Sep 2025).

1. Origins, scope, and problem setting

FEWT was proposed for humanoid robot perception in the context of embodied intelligence, where robust perception must support manipulation in dynamic, real-world environments. The framework addresses shortcomings in prior approaches, notably the ACT baseline, that primarily rely on time-domain or spatial feature extraction. Its stated objective is to improve perception representation by combining multi-scale wavelet decomposition with Transformer-based policy learning (Huang et al., 14 Sep 2025).

The framework is embedded in a complete imitation learning setting rather than being only a perception-side feature extractor. The reported system includes a humanoid robot, an exoskeleton-style teleoperation cabin, and an anthropomorphic data-collection pipeline. The key modalities are visual data, proprioceptive data, IMU data, and action sequences. More specifically, the inputs are RGB images from 3 cameras, robot joint positions, linear velocity and acceleration of the mobile base, and action sequences used for imitation learning (Huang et al., 14 Sep 2025).

A common simplification is to treat FEWT as a purely image-based wavelet Transformer. The architecture as described is broader: it couples visual perception with temporal sequence modeling and action prediction inside a Conditional Variational Autoencoder (CVAE) policy. This suggests that the term “perception” in FEWT refers to a multimodal representation used for downstream control, not only to camera-side feature extraction.

2. Architectural organization within the CVAE policy

FEWT builds upon the CVAE framework. In this formulation, the encoder compresses action sequences, robotic joint data, and IMU data into a low-dimensional latent variable $z$, while the decoder uses multi-view visual inputs, joint states, IMU data, and $z$ to predict action trajectories in an autoregressive manner (Huang et al., 14 Sep 2025).

The architecture is organized around two enhancement paths. The visual backbone processes images through ResNet18 augmented with FE-EMA, thereby extracting fused spatial features per view. The temporal backbone applies TS-DWT to time series such as actions, robot states, and IMU signals to obtain frequency-aware temporal encodings. These representations are then combined, augmented with positional encoding, and provided to the Transformer encoder-decoder stack. During inference, the latent vector $z$ is set to $0$ (Huang et al., 14 Sep 2025).

This organization makes FEWT a hybrid multimodal policy architecture. It does not replace the Transformer with a wavelet-only mechanism; instead, it inserts wavelet-based spectral structure into both the visual and temporal feature pathways before multimodal fusion. A plausible implication is that FEWT inherits the sequence modeling capacity of Transformer policies while reducing the burden of discovering multi-scale spectral structure purely from raw time-domain inputs.

3. Frequency-Enhanced Efficient Multi-Scale Attention (FE-EMA)

FE-EMA extends the Efficient Multi-Scale Attention (EMA) module by explicitly incorporating frequency-domain features via wavelet transform in addition to standard time-domain attention maps. The stated rationale is that wavelet decomposition captures informative structure at multiple scales and provides robustness to noise (Huang et al., 14 Sep 2025).

The module begins from a wavelet decomposition of an input feature map $X$:

$$X = \sum_{j\in \mathbb{Z}}\sum_{k\in \mathbb{Z}} cA_{j,k}\,\phi_{j,k}(x) + \sum_{i=1}^3\sum_{j\in \mathbb{Z}}\sum_{k\in \mathbb{Z}} cD_{i,j,k}\,\psi_{i,j,k}(x).$$

Here, $cA_{j,k}$ denotes the low-frequency approximation coefficient at scale $j$ and location $k$, and $cD_{i,j,k}$ denotes the high-frequency detail coefficients in horizontal, vertical, and diagonal directions. For implementation, FEWT uses Haar wavelet convolution kernels:

$z$0

For 2D images, the DWT yields four components, written as $z$1 (Huang et al., 14 Sep 2025).

FE-EMA computes attention-based time-domain features $z$2 and wavelet-decomposed frequency-domain features $z$3, then combines them through an adaptive scalar weight:

$z$4

$z$5

The fusion is performed inside a residual block, and grouped convolution is applied only to downsampled DWT-processed features to reduce FLOPs and improve efficiency (Huang et al., 14 Sep 2025).

Functionally, FE-EMA adaptively balances time-domain and frequency-domain information at each stage. The stated effect is to capture feature information across various scales effectively and thereby enhance model robustness. The paper also reports that FE-EMA plus ResNet18 keeps compute efficiency nearly identical to baseline EMA plus ResNet18 (Huang et al., 14 Sep 2025).

4. Time-Series Discrete Wavelet Transform (TS-DWT)

TS-DWT is the temporal counterpart to FE-EMA. Its purpose is to decompose one-dimensional temporal sequences so that the policy can use both steady low-frequency and dynamic high-frequency components during action prediction (Huang et al., 14 Sep 2025).

The module receives a time-series tensor, such as an action or observation sequence, and applies DWT to each one-dimensional sequence, producing low-frequency coefficients $z$6 and high-frequency coefficients $z$7. These coefficients are then upsampled via linear projection, concatenated, and fused with a 1D convolution. A fully connected layer generates frequency-domain attention weights, and the learned weights are multiplied with the original sequence tensor to produce an enhanced, frequency-aware temporal representation (Huang et al., 14 Sep 2025).

The role of TS-DWT in FEWT is therefore not merely denoising or compression. It explicitly supplies the Transformer with a decomposition of temporal content into slow and fast components. In the formulation given in the paper, these processed features are presented to the Transformer encoder-decoder so that the policy can exploit both stable trends and rapid variations. This suggests that FEWT treats wavelet analysis as a structural prior for policy learning rather than as a post hoc signal-processing stage.

A frequent misconception is that wavelet modules in robot learning are confined to image processing. FEWT directly applies wavelet decomposition to time-series tensors and uses the resulting representation for action-related temporal encoding, making the wavelet contribution explicitly multimodal (Huang et al., 14 Sep 2025).

5. Multimodal fusion, policy learning, and interpretive properties

After FE-EMA extracts fused spatial features and TS-DWT yields frequency-aware temporal encodings, FEWT combines these modalities with positional encoding and feeds them to a Transformer encoder-decoder stack. The decoder then autoregressively generates action sequences conditioned on the multimodal state and the latent variable $z$8 (Huang et al., 14 Sep 2025).

This multimodal arrangement is significant because the visual stream and the temporal stream are enhanced through different but conceptually aligned frequency mechanisms. The visual path uses wavelet decomposition to complement attention maps with cross-spatial and frequency-domain information, while the temporal path decomposes sequences into low- and high-frequency components and reweights them through learned frequency-domain attention. The resulting policy therefore operates on fused representations that are already structured by scale and spectral content before standard Transformer sequence modeling (Huang et al., 14 Sep 2025).

The paper provides two forms of interpretive evidence. First, Grad-CAM visualizations indicate that FE-EMA enables transformer policies to focus more accurately on task-critical visual regions. Second, the adaptive weight $z$9 used for time/frequency fusion remains well balanced during training. These observations do not establish a formal interpretability guarantee, but they are presented as evidence that the frequency-enhanced fusion mechanism is active and dynamically regulated rather than collapsing to one branch (Huang et al., 14 Sep 2025).

6. Empirical evaluation and reported gains

The simulation evaluation covers Cube Transfer, Bimanual Insertion, Building Blocks, Drawer Storage, Lifted Curtain, and Strip Transfer. The primary benchmark is the ACT baseline. According to the reported ablations, adding FE-EMA and TS-DWT each improves success rate, while combining both into FEWT produces the largest gains. FEWT is reported to achieve up to 30% improvement over the ACT baseline in simulation tasks. One example given is Cube Transfer in the transfer phase, where ACT achieves 64% and FEWT achieves 94% (Huang et al., 14 Sep 2025).

The real-world evaluation is conducted on physical humanoid tasks including Charm Splicing, Throw Paper, Drawer Storage, and Doll Storage. FEWT is reported to improve over ACT by 6–12% in real-world bimanual manipulation tasks. The examples given are Drawer Storage, from ACT 74% to FEWT 84%, and Doll Storage, from ACT 70% to FEWT 82%. The paper also states that incorporating IMU data into FEWT further improved motion control accuracy in dynamic tasks (Huang et al., 14 Sep 2025).

The comparative study additionally reports that FEWT outperforms diffusion-policy variants denoted DP (DDPM, DDIM) and ACT in all evaluated settings. The authors further state that FLOPs remain almost unchanged relative to the baseline EMA plus ResNet18 configuration, supporting the claim that the framework improves robustness without a large computational penalty (Huang et al., 14 Sep 2025).

These findings are presented as evidence that explicit fusion of time-domain and frequency-domain information can improve both simulated and physical manipulation policies. A plausible implication is that the gains are strongest when task success depends on multiscale visual cues, dynamic motion states, and temporal coordination.

7. Position within the broader literature on frequency- and wavelet-enhanced Transformers

FEWT belongs to a broader class of architectures that explicitly integrate Fourier- or wavelet-domain processing into Transformer pipelines. In RGB-event tracking, “FreqTrack” introduces a Spectral Enhancement Transformer (SET) with dynamic Fourier filtering and a Wavelet Edge Refinement (WER) module with learnable Haar wavelets to improve modeling under fast motion and low light; it reports a leading precision of 76.6% on COESOT (You et al., 16 Apr 2026). In long-term forecasting, “FEDformer” combines seasonal-trend decomposition with frequency enhanced blocks and wavelet-enhanced alternatives, and reports linear complexity together with error reductions of 14.8% for multivariate and 22.6% for univariate forecasting (Zhou et al., 2022).

Other instances of the same design tendency appear across domains. “WaveFormer” for biomedical signal classification uses DWT in both embedding construction and Dynamic Wavelet Positional Encoding, and reports the highest accuracy on 7 out of 8 datasets (Irani et al., 12 Feb 2026). “FEFormer” for volumetric medical image segmentation integrates FFT- and wavelet-based modules—FDSA, FGMLP, WAFF, and FCSB—to capture global context, fine details, and semantically consistent encoder-decoder fusion (Yang et al., 12 May 2026). “AWGformer” further generalizes the idea by learning wavelet bases, decomposition levels, cross-scale coupling, and frequency-aware attention heads for multiresolution forecasting (Li, 28 Jan 2026).

Within this landscape, FEWT is distinctive in targeting humanoid robot imitation learning rather than tracking, forecasting, classification, or segmentation. Its particular contribution is the joint use of wavelet-enhanced visual attention and wavelet-enhanced temporal modeling inside a CVAE Transformer policy. This suggests that FEWT can be understood as a robotics-specific instance of a wider methodological pattern: explicit frequency decomposition is used to compensate for the limitations of standard self-attention when data are multi-scale, non-stationary, or detail-sensitive.