Fremer: Frequency-Domain Transformer for Cloud Workloads

• Fremer is a lightweight frequency-domain Transformer that predicts cloud workload series with high efficiency and accuracy by operating in the spectral domain.
• It aligns input and target frequency resolutions using Learnable Linear Padding and employs frequency filters and frequency-domain normalization to extract essential periodic signals.
• By modeling frequency combinations with Complex-valued Spectrum Attention, Fremer reduces computational costs while outperforming state-of-the-art methods in cloud workload forecasting.

Fremer is a lightweight, frequency-domain Transformer for workload forecasting in cloud services. It is designed for settings in which forecasting supports auto-scaling, scheduling, and provisioning, and in which both predictive accuracy and computational efficiency are operational constraints. The model is motivated by two observations: cloud workloads are highly periodic and often contain long-range temporal structure that is easier to model in the frequency domain than in the raw time domain, while conventional Transformer-based forecasting models are often too expensive for large-scale cloud deployment because self-attention has quadratic complexity in sequence length. The method is introduced in "Fremer: Lightweight and Effective Frequency Transformer for Workload Forecasting in Cloud Services" (Chen et al., 17 Jul 2025).

1. Problem formulation and design rationale

The forecasting target is a workload series

$\mathbf{X}=\{x_0,x_1,\dots,x_{L-1}\},$

with the task of predicting future values

$$\mathbf{\hat{Y}}=\{x_{L},x_{L+1},\dots,x_{L+T-1}\},$$

where $L$ is the look-back window and $T$ is the forecast horizon (Chen et al., 17 Jul 2025).

The motivating deployment context is cloud operations. In cloud systems, workload forecasting predicts future resource demand or request volume for instances, services, or clusters so that operators can proactively autoscale, schedule, or provision resources. The paper emphasizes that ByteDance’s PaaS environment alone requires over 100,000 forecasting tasks per hour, making the joint optimization of accuracy and efficiency a central systems requirement.

The rationale for moving from the time domain to the frequency domain is tied to the structure of cloud workload traces. Workloads commonly show hourly, daily, and weekly periodicity, often mixed with noise and local shifts. In the time domain, these factors are entangled; in the frequency domain, periodic components and harmonics appear more explicitly, and useful information is concentrated in only a few frequencies. This suggests that spectral representations can be simultaneously more compact and more interpretable for this class of forecasting task.

The paper identifies two specific obstacles that earlier frequency-aware Transformers had not solved well. The first is frequency mis-alignment: DFT frequency grids depend on sequence length, so the spectrum of the observed history and the spectrum of the full sequence to be predicted can have different frequency resolutions. The second is frequency combination modeling: important periodic information often appears as combinations or harmonics rather than isolated single frequency points, so point-wise frequency attention is insufficient. These two issues define the architectural problem that Fremer is designed to solve.

2. Architectural composition

Fremer is an encoder-only Transformer redesigned for the frequency domain. Rather than forecasting directly in time, it forecasts the spectrum of the complete sequence from the spectrum of the input sequence and then transforms the result back to the time domain (Chen et al., 17 Jul 2025).

Its pipeline consists of six stages. First, Learnable Linear Padding (LLP) pads the input from length $L$ to $L+T$. Second, the padded series is converted with rFFT to the frequency domain. Third, Frequency Filters remove extreme high-frequency noise and separate low-frequency trend components. Fourth, Frequency-Reversible Instance Norm (F-RIN) normalizes the spectrum. Fifth, Complex-valued Spectrum Attention (CSA) models relationships among frequency combinations. Sixth, the output spectrum is mapped back by irFFT, and the last $T$ points are returned as the forecast.

A defining property of the architecture is that it is channel-independent. The paper states that this is important because the workload of one instance is often weakly correlated with others, making the model more generalizable to unseen instances. The empirical discussion further notes that channel-independent methods tend to outperform methods that explicitly model inter-instance correlations, because workloads across instances are often relatively autonomous in these datasets.

The model’s filtering strategy is specific. The LPF (Low-Pass Filter) removes the highest-frequency noise by setting the highest 1% of frequencies to $0+0j$. The HPF (High-Pass Filter) retains the high-frequency portion and is used to control overfitting, because the model may over-focus on low-frequency dominant components. A specific design choice is to keep the lowest 3% of frequencies to preserve trend information, but exclude this low-frequency part from the backbone input to avoid overfitting; after prediction, the low-frequency components are concatenated back into the final output.

This design places Fremer among frequency-domain forecasting models, but with an explicit emphasis on deployment-scale efficiency. A plausible implication is that the model’s architectural choices are not merely representational; they are also resource-allocation decisions intended to preserve spectral structure while reducing the computational burden of sequence modeling.

3. Mathematical formulation and computational properties

The spectral basis of Fremer is the discrete Fourier transform. The paper gives the DFT as

$$\mathbf{F}[k]=\sum_{n=0}^{L-1}\mathbf{X}[n]e^{-2\pi i\frac{kn}{L}},\ k=0,1,\dots,L-1,$$

and the inverse DFT as

$$\mathbf{X}[n]=\frac{1}{L}\sum_{k=0}^{L-1}\mathbf{F}[k]e^{2\pi i\frac{kn}{L}},\ n=0,1,\dots,L-1.$$

In practice, the model uses the real FFT to exploit Hermitian symmetry and reduce cost (Chen et al., 17 Jul 2025).

To address frequency-grid misalignment, Fremer introduces Learnable Linear Padding: $$\mathbf{\hat{Y}}=\{x_{L},x_{L+1},\dots,x_{L+T-1}\},$$0 where $$\mathbf{\hat{Y}}=\{x_{L},x_{L+1},\dots,x_{L+T-1}\},$$1, $$\mathbf{\hat{Y}}=\{x_{L},x_{L+1},\dots,x_{L+T-1}\},$$2, $$\mathbf{\hat{Y}}=\{x_{L},x_{L+1},\dots,x_{L+T-1}\},$$3, and $$\mathbf{\hat{Y}}=\{x_{L},x_{L+1},\dots,x_{L+T-1}\},$$4. By padding to the same length as the target full series, the frequency resolution of the input spectrum aligns with the spectrum of the complete sequence, making it possible to model the same frequencies consistently.

For normalization, Fremer adapts RevIN to the spectral domain through F-RIN: $$\mathbf{\hat{Y}}=\{x_{L},x_{L+1},\dots,x_{L+T-1}\},$$5 The standard deviation is computed from the magnitude $$\mathbf{\hat{Y}}=\{x_{L},x_{L+1},\dots,x_{L+T-1}\},$$6, and the stated purpose is to align spectra from series with different global characteristics.

The central modeling block is Complex-valued Spectrum Attention. After LLP, FFT, filtering, and F-RIN, the spectrum is projected into a smaller frequency-combination space

$$\mathbf{\hat{Y}}=\{x_{L},x_{L+1},\dots,x_{L+T-1}\},$$7

via a trainable linear projection $$\mathbf{\hat{Y}}=\{x_{L},x_{L+1},\dots,x_{L+T-1}\},$$8. For each head $$\mathbf{\hat{Y}}=\{x_{L},x_{L+1},\dots,x_{L+T-1}\},$$9,

$L$0

where $L$1. The attention rule is

$L$2

The multi-head output is

$L$3

with $L$4.

The stated intuition is that a single frequency point rarely has enough semantic meaning, whereas combinations of frequencies encode periodic patterns and harmonics. CSA therefore attends over frequency combinations rather than individual frequencies. The paper also notes that, compared with the earlier Frequency Enhanced Attention in FEDformer, CSA is channel-independent and computes attention over combinations, not sampled points.

The computational argument follows directly from the reduced attention space. Standard dot-product attention has time and space complexity $L$5. Fremer reduces attention complexity to

$L$6

where $L$7 is the number of heads. Because $L$8 is much smaller than $L$9 and because of inverted multi-head projection, the model is described as far cheaper than standard attention.

4. Data resources and evaluation protocol

The empirical study uses seven workload datasets: four ByteDance cloud workload datasets and three public benchmark datasets from Materna-Workload-Traces (Chen et al., 17 Jul 2025).

Dataset	Service or source	Key characteristics
FaaS	Function-as-a-Service	QPS, 226 instances, length 2305, 10-min, 2022-04-02 to 2022-04-18
IaaS	Infrastructure-as-a-Service	CPU usage, 93 instances, length 3456, 10-min, 2023-06-30 to 2023-07-24
PaaS	Platform-as-a-Service	CPU usage in milli-cores, 426 instances, length 7776, 10-min, 2024-09-01 to 2024-10-24
RDS	Relational Database Service	QPS, 1113 instances, length 6624, 10-min, 2024-08-16 to 2024-09-30
MT-1	Materna-Workload-Traces	413 instances, length 8352, 5-min
MT-2	Materna-Workload-Traces	402 instances, length 8928, 5-min
MT-3	Materna-Workload-Traces	371 instances, length 10368, 5-min

The public datasets contain CPU usage traces from a German distributed cloud data center, and VMs with more than 2% missing rate are filtered out. The four ByteDance datasets are described as open-source workload datasets derived from ByteDance’s cloud services and encompassing workload data from thousands of computing instances.

The evaluation platform is the Time Series Forecasting Benchmark (TFB), with all baselines taken from TFB and default settings retained. The training/validation/test split is 7:1:2 by time span. The input window is 5 days: 1440 points for 5-min datasets and 720 points for 10-min datasets. The forecast horizon is 1 day: 288 points for 5-min datasets and 144 points for 10-min datasets. Training uses Adam, learning rate 1e-3, MSE loss, batch size 32, and 20 epochs, on NVIDIA A100-SXM 80GB GPUs. The paper states that Fremer uses only a limited hyperparameter search, mainly over the frequency filter threshold.

The comparison set covers several architectural families. Transformer-based baselines are Fredformer, PDF, iTransformer, PatchTST, Crossformer, FEDformer, and Informer. MLP-based baselines are FITS, DLinear, and NLinear. CNN-based baselines are MICN and TimesNet. FECAM is listed as attention-based.

The metrics are MSE, MAE, and SMAPE. Because raw scales differ greatly across datasets, MSE and MAE are computed on normalized data, while SMAPE is computed on raw data. The normalization uses training-set mean and standard deviation to avoid leakage.

5. Empirical performance and deployment behavior

On the seven workload datasets, Fremer is reported as the best model on most datasets and usually the best overall (Chen et al., 17 Jul 2025). Averaged across datasets, it improves over the strongest baseline by 2.9% in MSE, 2.5% in MAE, and 3.5% in SMAPE. The abstract also reports an average improvement over SOTA models of 5.5% in MSE, 4.7% in MAE, and 8.6% in SMAPE. The paper explicitly states that these two sets of percentages reflect different comparison aggregates described in the paper.

Representative dataset-level results include PaaS with 0.062 MSE / 0.137 MAE / 5.329% SMAPE, FaaS with 0.289 MSE / 0.314 MAE / 9.240% SMAPE, RDS with 1.292 MSE / 0.364 MAE / 8.663% SMAPE, IaaS with 0.708 MSE / 0.556 MAE / 16.249% SMAPE, MT-1 with 15.408 MSE / 0.433 MAE / 18.828% SMAPE, MT-2 with 6.543 MSE / 0.287 MAE / 14.360% SMAPE, and MT-3 with 28.062 MSE / 0.699 MAE / 21.411% SMAPE.

Efficiency is a co-equal empirical claim. For IaaS, Fremer has 61.67 ms/iter training time, 34.39 ms/iter inference time, and 0.57M parameters. For RDS, it has 105.23 ms/iter training time, 27.87 ms/iter inference time, and 0.59M parameters. Relative to PatchTST, on IaaS it reduces training time by 82.2%, inference time by 28.0%, and parameter count by 91.7%; on RDS it reduces training time by 81.2%, inference time by 13.1%, and parameter count by 86.5%.

The paper further validates practical value through a Kubernetes Horizontal Pod Autoscaler (HPA) proactive scaling simulation. The setup uses a 5-day history to predict the next day’s workload, replays the predicted workload in a 24-hour simulation, and uses a ByteDance FaaS function instance. The metrics are average latency, tail latency, timeout rate, and average pod count.

Model	Latency and timeout	AvePod
PatchTST	Ave-Lat 1.017, 99.9-Lat 10.0, 99-Lat 3.644, 90-Lat 1.651, Timeout Rate 0.132%	22.479
DLinear	Ave-Lat 1.081, 99.9-Lat 10.0, 99-Lat 4.108, 90-Lat 1.779, Timeout Rate 0.22%	19.25
FITS	Ave-Lat 1.05, 99.9-Lat 10.0, 99-Lat 4.101, 90-Lat 1.704, Timeout Rate 0.279%	21.889
Naïve HPA	Ave-Lat 0.996, 99.9-Lat 10.0, 99-Lat 3.764, 90-Lat 1.56, Timeout Rate 0.386%	29.181
Ideal	Ave-Lat 0.789, 99.9-Lat 3.513, 99-Lat 2.063, 90-Lat 1.206, Timeout Rate 0.026%	21.382
Fremer	Ave-Lat 0.826, 99.9-Lat 10.0, 99-Lat 2.292, 90-Lat 1.261, Timeout Rate 0.102%	21.951

In this simulation, Fremer reduces average latency by 18.78% compared to PatchTST and reduces resource consumption by 2.35% on average. The paper interprets this as evidence that better forecasting translates into better service quality and lower operational cost in real cloud scheduling.

6. Ablation findings, generalization, and stated boundaries

The ablation study attributes measurable value to each major module (Chen et al., 17 Jul 2025). Removing LLP hurts performance significantly, which the paper uses to confirm the importance of frequency alignment. Removing F-Filter hurts especially on noisy datasets. Removing F-RIN degrades results, indicating that normalization in the frequency domain is beneficial. Replacing CSA with attention over individual frequency components lowers performance, which the paper presents as evidence that modeling frequency combinations is valuable.

The model is also described as robust for multi-period series. The paper states that Fremer handles workload series with multiple intertwined periodicities; that sensitivity analysis shows strong performance when the input window includes at least one day of history; that the model can capture both hourly and sub-hourly periodic elements using different attention heads; and that the best performance often occurs when $T$0, showing that a compressed frequency-combination representation is sufficient. This robustness is reinforced by multi-head visualization, where different heads attend to different periodic structures.

Generalization is evaluated in both intra-dataset transfer and cross-dataset transfer. Intra-dataset transfer trains on some instances and tests on unseen instances from the same dataset. Cross-dataset transfer trains on one dataset and tests on another. Fremer achieves the best performance in both settings, demonstrating strong generalization to unseen workloads and even zero-shot transfer across different cloud service types. In operational terms, this is relevant because new services or new instances often appear with little labeled history.

The paper does not emphasize a formal limitations section, but it implicitly suggests several boundaries. Fremer is designed around periodic workloads and may be less ideal for highly irregular, non-periodic series. The model still uses FFT-based operations and a frequency-combination attention mechanism, so it is lightweight but not free of spectral-processing overhead. The current work focuses on workload forecasting and one Kubernetes autoscaling case study; broader production deployment would need further validation across more systems and control policies.

Taken together, the paper’s main contributions are a new frequency-domain Transformer architecture for cloud workload forecasting, Learnable Linear Padding for frequency-resolution alignment, Complex-valued Spectrum Attention for efficient modeling of frequency combinations, frequency filters and frequency-domain normalization to reduce noise and overfitting, four open-source ByteDance cloud workload datasets, and experiments showing state-of-the-art accuracy, lower compute cost, strong transferability, and practical gains in Kubernetes autoscaling. A plausible implication is that Fremer occupies a specific methodological niche: it is not a general-purpose replacement for all time-series Transformers, but a specialized spectral architecture for cloud workloads whose dominant inductive bias is periodic structure.