---
title: 'Fremer: Frequency-Domain Transformer for Cloud Workloads'
url: https://www.emergentmind.com/topics/fremer
type: topic
---

# Fremer: Frequency-Domain Transformer for Cloud Workloads

Fremer is a lightweight, frequency-domain Transformer for workload forecasting in cloud services. It is designed for settings in which forecasting supports auto-scaling, scheduling, and provisioning, and in which both predictive accuracy and computational efficiency are operational constraints. The model is motivated by two observations: cloud workloads are highly periodic and often contain long-range temporal structure that is easier to model in the frequency domain than in the raw time domain, while conventional Transformer-based forecasting models are often too expensive for large-scale cloud deployment because self-attention has quadratic complexity in sequence length. The method is introduced in "Fremer: Lightweight and Effective Frequency Transformer for Workload Forecasting in Cloud Services" [2507.12908].

## 1. Problem formulation and design rationale

The forecasting target is a workload series
\[
\mathbf{X}=\{x_0,x_1,\dots,x_{L-1}\},
\]
with the task of predicting future values
\[
\mathbf{\hat{Y}}=\{x_{L},x_{L+1},\dots,x_{L+T-1}\},
\]
where \(L\) is the look-back window and \(T\) is the forecast horizon [2507.12908].

The motivating deployment context is cloud operations. In cloud systems, workload forecasting predicts future resource demand or request volume for instances, services, or clusters so that operators can proactively autoscale, schedule, or provision resources. The paper emphasizes that ByteDance’s PaaS environment alone requires over 100,000 forecasting tasks per hour, making the joint optimization of accuracy and efficiency a central systems requirement.

The rationale for moving from the time domain to the frequency domain is tied to the structure of cloud workload traces. Workloads commonly show hourly, daily, and weekly periodicity, often mixed with noise and local shifts. In the time domain, these factors are entangled; in the frequency domain, periodic components and harmonics appear more explicitly, and useful information is concentrated in only a few frequencies. This suggests that spectral representations can be simultaneously more compact and more interpretable for this class of forecasting task.

The paper identifies two specific obstacles that earlier frequency-aware Transformers had not solved well. The first is **frequency mis-alignment**: DFT frequency grids depend on sequence length, so the spectrum of the observed history and the spectrum of the full sequence to be predicted can have different frequency resolutions. The second is **frequency combination modeling**: important periodic information often appears as combinations or harmonics rather than isolated single frequency points, so point-wise frequency attention is insufficient. These two issues define the architectural problem that Fremer is designed to solve.

## 2. Architectural composition

Fremer is an encoder-only Transformer redesigned for the frequency domain. Rather than forecasting directly in time, it forecasts the spectrum of the complete sequence from the spectrum of the input sequence and then transforms the result back to the time domain [2507.12908].

Its pipeline consists of six stages. First, **Learnable Linear Padding (LLP)** pads the input from length \(L\) to \(L+T\). Second, the padded series is converted with **rFFT** to the frequency domain. Third, **Frequency Filters** remove extreme high-frequency noise and separate low-frequency trend components. Fourth, **Frequency-Reversible Instance Norm (F-RIN)** normalizes the spectrum. Fifth, **Complex-valued Spectrum Attention (CSA)** models relationships among frequency combinations. Sixth, the output spectrum is mapped back by **irFFT**, and the last \(T\) points are returned as the forecast.

A defining property of the architecture is that it is **channel-independent**. The paper states that this is important because the workload of one instance is often weakly correlated with others, making the model more generalizable to unseen instances. The empirical discussion further notes that channel-independent methods tend to outperform methods that explicitly model inter-instance correlations, because workloads across instances are often relatively autonomous in these datasets.

The model’s filtering strategy is specific. The **LPF (Low-Pass Filter)** removes the highest-frequency noise by setting the highest 1% of frequencies to \(0+0j\). The **HPF (High-Pass Filter)** retains the high-frequency portion and is used to control overfitting, because the model may over-focus on low-frequency dominant components. A specific design choice is to keep the lowest 3% of frequencies to preserve trend information, but exclude this low-frequency part from the backbone input to avoid overfitting; after prediction, the low-frequency components are concatenated back into the final output.

This design places Fremer among frequency-domain forecasting models, but with an explicit emphasis on deployment-scale efficiency. A plausible implication is that the model’s architectural choices are not merely representational; they are also resource-allocation decisions intended to preserve spectral structure while reducing the computational burden of sequence modeling.

## 3. Mathematical formulation and computational properties

The spectral basis of Fremer is the discrete Fourier transform. The paper gives the DFT as
\[
\mathbf{F}[k]=\sum_{n=0}^{L-1}\mathbf{X}[n]e^{-2\pi i\frac{kn}{L}},\ k=0,1,\dots,L-1,
\]
and the inverse DFT as
\[
\mathbf{X}[n]=\frac{1}{L}\sum_{k=0}^{L-1}\mathbf{F}[k]e^{2\pi i\frac{kn}{L}},\ n=0,1,\dots,L-1.
\]
In practice, the model uses the real FFT to exploit Hermitian symmetry and reduce cost [2507.12908].

To address frequency-grid misalignment, Fremer introduces **Learnable Linear Padding**:
\[
\mathbf{X}_p=concat(\mathbf{X}, \mathbf{W}^T\mathbf{X}+b),
\]
where \(\mathbf{X}\in \mathbb{R}^{L}\), \(\mathbf{W}\in \mathbb{R}^{L\times T}\), \(b\in \mathbb{R}^{T}\), and \(\mathbf{X}_p\in \mathbb{R}^{L+T}\). By padding to the same length as the target full series, the frequency resolution of the input spectrum aligns with the spectrum of the complete sequence, making it possible to model the same frequencies consistently.

For normalization, Fremer adapts RevIN to the spectral domain through **F-RIN**:
\[
\mathbf{F}_r = \frac{\mathbf{F}_f-mean(\mathbf{F}_f)}{std(|\mathbf{F}_f|)}.
\]
The standard deviation is computed from the magnitude \(|\mathbf{F}_f|\), and the stated purpose is to align spectra from series with different global characteristics.

The central modeling block is **Complex-valued Spectrum Attention**. After LLP, FFT, filtering, and F-RIN, the spectrum is projected into a smaller frequency-combination space
\[
\mathbf{F}_c\in\mathbb{C}^{L^\prime},
\]
via a trainable linear projection \(\mathbf{W}_{proj}\in \mathbb{C}^{L^\prime\times (\lfloor\frac{L+T}{2}\rfloor+1-L_f)}\). For each head \(h\),
\[
\mathbf{Q}_h=\mathbf{F}_c^T\mathbf{W}_h^Q,\quad
\mathbf{K}_h=\mathbf{F}_c^T\mathbf{W}_h^K,\quad
\mathbf{V}_h=\mathbf{F}_c^T\mathbf{W}_h^V,
\]
where \(\mathbf{W}_h^Q, \mathbf{W}_h^K, \mathbf{W}_h^V\in \mathbb{C}^{L^\prime\times l}\). The attention rule is
\[
Attention(\mathbf{Q}_h, \mathbf{K}_h, \mathbf{V}_h)=Softmax (|\mathbf{Q}_h\mathbf{K}_h^T|)\mathbf{V}_h.
\]
The multi-head output is
\[
\begin{aligned}
\text{head}_h &= Attention(\mathbf{Q}_h, \mathbf{K}_h, \mathbf{V}_h),\\
\text{CSA}(\mathbf{F}_c)&=Concat(\text{head}_1, \dots, \text{head}_H)^T\mathbf{W}_O,
\end{aligned}
\]
with \(\mathbf{W}_O\in \mathbb{C}^{hl\times L^\prime}\).

The stated intuition is that a single frequency point rarely has enough semantic meaning, whereas combinations of frequencies encode periodic patterns and harmonics. CSA therefore attends over frequency combinations rather than individual frequencies. The paper also notes that, compared with the earlier Frequency Enhanced Attention in FEDformer, CSA is channel-independent and computes attention over combinations, not sampled points.

The computational argument follows directly from the reduced attention space. Standard dot-product attention has time and space complexity \(O(L^2)\). Fremer reduces attention complexity to
\[
O\!\left(\frac{(L^\prime)^2}{H}\right),
\]
where \(H\) is the number of heads. Because \(L^\prime\) is much smaller than \(L\) and because of inverted multi-head projection, the model is described as far cheaper than standard attention.

## 4. Data resources and evaluation protocol

The empirical study uses seven workload datasets: four ByteDance cloud workload datasets and three public benchmark datasets from Materna-Workload-Traces [2507.12908].

| Dataset | Service or source | Key characteristics |
|---|---|---|
| FaaS | Function-as-a-Service | QPS, 226 instances, length 2305, 10-min, 2022-04-02 to 2022-04-18 |
| IaaS | Infrastructure-as-a-Service | CPU usage, 93 instances, length 3456, 10-min, 2023-06-30 to 2023-07-24 |
| PaaS | Platform-as-a-Service | CPU usage in milli-cores, 426 instances, length 7776, 10-min, 2024-09-01 to 2024-10-24 |
| RDS | Relational Database Service | QPS, 1113 instances, length 6624, 10-min, 2024-08-16 to 2024-09-30 |
| MT-1 | Materna-Workload-Traces | 413 instances, length 8352, 5-min |
| MT-2 | Materna-Workload-Traces | 402 instances, length 8928, 5-min |
| MT-3 | Materna-Workload-Traces | 371 instances, length 10368, 5-min |

The public datasets contain CPU usage traces from a German distributed cloud data center, and VMs with more than 2% missing rate are filtered out. The four ByteDance datasets are described as open-source workload datasets derived from ByteDance’s cloud services and encompassing workload data from thousands of computing instances.

The evaluation platform is the **Time Series Forecasting Benchmark (TFB)**, with all baselines taken from TFB and default settings retained. The training/validation/test split is **7:1:2** by time span. The input window is **5 days**: 1440 points for 5-min datasets and 720 points for 10-min datasets. The forecast horizon is **1 day**: 288 points for 5-min datasets and 144 points for 10-min datasets. Training uses **Adam**, learning rate **1e-3**, **MSE loss**, batch size **32**, and **20** epochs, on **NVIDIA A100-SXM 80GB GPUs**. The paper states that Fremer uses only a limited hyperparameter search, mainly over the frequency filter threshold.

The comparison set covers several architectural families. Transformer-based baselines are Fredformer, PDF, iTransformer, PatchTST, Crossformer, FEDformer, and Informer. MLP-based baselines are FITS, DLinear, and NLinear. CNN-based baselines are MICN and TimesNet. FECAM is listed as attention-based.

The metrics are **MSE**, **MAE**, and **SMAPE**. Because raw scales differ greatly across datasets, MSE and MAE are computed on normalized data, while SMAPE is computed on raw data. The normalization uses training-set mean and standard deviation to avoid leakage.

## 5. Empirical performance and deployment behavior

On the seven workload datasets, Fremer is reported as the best model on most datasets and usually the best overall [2507.12908]. Averaged across datasets, it improves over the strongest baseline by **2.9% in MSE**, **2.5% in MAE**, and **3.5% in SMAPE**. The abstract also reports an average improvement over SOTA models of **5.5% in MSE**, **4.7% in MAE**, and **8.6% in SMAPE**. The paper explicitly states that these two sets of percentages reflect different comparison aggregates described in the paper.

Representative dataset-level results include **PaaS** with **0.062 MSE / 0.137 MAE / 5.329% SMAPE**, **FaaS** with **0.289 MSE / 0.314 MAE / 9.240% SMAPE**, **RDS** with **1.292 MSE / 0.364 MAE / 8.663% SMAPE**, **IaaS** with **0.708 MSE / 0.556 MAE / 16.249% SMAPE**, **MT-1** with **15.408 MSE / 0.433 MAE / 18.828% SMAPE**, **MT-2** with **6.543 MSE / 0.287 MAE / 14.360% SMAPE**, and **MT-3** with **28.062 MSE / 0.699 MAE / 21.411% SMAPE**.

Efficiency is a co-equal empirical claim. For **IaaS**, Fremer has **61.67 ms/iter** training time, **34.39 ms/iter** inference time, and **0.57M** parameters. For **RDS**, it has **105.23 ms/iter** training time, **27.87 ms/iter** inference time, and **0.59M** parameters. Relative to PatchTST, on IaaS it reduces training time by **82.2%**, inference time by **28.0%**, and parameter count by **91.7%**; on RDS it reduces training time by **81.2%**, inference time by **13.1%**, and parameter count by **86.5%**.

The paper further validates practical value through a **Kubernetes Horizontal Pod Autoscaler (HPA)** proactive scaling simulation. The setup uses a **5-day history** to predict the next day’s workload, replays the predicted workload in a **24-hour simulation**, and uses a ByteDance FaaS function instance. The metrics are average latency, tail latency, timeout rate, and average pod count.

| Model | Latency and timeout | AvePod |
|---|---|---:|
| PatchTST | Ave-Lat 1.017, 99.9-Lat 10.0, 99-Lat 3.644, 90-Lat 1.651, Timeout Rate 0.132% | 22.479 |
| DLinear | Ave-Lat 1.081, 99.9-Lat 10.0, 99-Lat 4.108, 90-Lat 1.779, Timeout Rate 0.22% | 19.25 |
| FITS | Ave-Lat 1.05, 99.9-Lat 10.0, 99-Lat 4.101, 90-Lat 1.704, Timeout Rate 0.279% | 21.889 |
| Naïve HPA | Ave-Lat 0.996, 99.9-Lat 10.0, 99-Lat 3.764, 90-Lat 1.56, Timeout Rate 0.386% | 29.181 |
| Ideal | Ave-Lat 0.789, 99.9-Lat 3.513, 99-Lat 2.063, 90-Lat 1.206, Timeout Rate 0.026% | 21.382 |
| Fremer | Ave-Lat 0.826, 99.9-Lat 10.0, 99-Lat 2.292, 90-Lat 1.261, Timeout Rate 0.102% | 21.951 |

In this simulation, Fremer reduces **average latency by 18.78%** compared to PatchTST and reduces **resource consumption by 2.35%** on average. The paper interprets this as evidence that better forecasting translates into better service quality and lower operational cost in real cloud scheduling.

## 6. Ablation findings, generalization, and stated boundaries

The ablation study attributes measurable value to each major module [2507.12908]. Removing **LLP** hurts performance significantly, which the paper uses to confirm the importance of frequency alignment. Removing **F-Filter** hurts especially on noisy datasets. Removing **F-RIN** degrades results, indicating that normalization in the frequency domain is beneficial. Replacing **CSA** with attention over individual frequency components lowers performance, which the paper presents as evidence that modeling **frequency combinations** is valuable.

The model is also described as robust for multi-period series. The paper states that Fremer handles workload series with multiple intertwined periodicities; that sensitivity analysis shows strong performance when the input window includes at least one day of history; that the model can capture both hourly and sub-hourly periodic elements using different attention heads; and that the best performance often occurs when \(L' \approx L/5\), showing that a compressed frequency-combination representation is sufficient. This robustness is reinforced by multi-head visualization, where different heads attend to different periodic structures.

Generalization is evaluated in both **intra-dataset transfer** and **cross-dataset transfer**. Intra-dataset transfer trains on some instances and tests on unseen instances from the same dataset. Cross-dataset transfer trains on one dataset and tests on another. Fremer achieves the best performance in both settings, demonstrating strong generalization to unseen workloads and even zero-shot transfer across different cloud service types. In operational terms, this is relevant because new services or new instances often appear with little labeled history.

The paper does not emphasize a formal limitations section, but it implicitly suggests several boundaries. Fremer is designed around periodic workloads and may be less ideal for highly irregular, non-periodic series. The model still uses FFT-based operations and a frequency-combination attention mechanism, so it is lightweight but not free of spectral-processing overhead. The current work focuses on workload forecasting and one Kubernetes autoscaling case study; broader production deployment would need further validation across more systems and control policies.

Taken together, the paper’s main contributions are a new frequency-domain Transformer architecture for cloud workload forecasting, **Learnable Linear Padding** for frequency-resolution alignment, **Complex-valued Spectrum Attention** for efficient modeling of frequency combinations, frequency filters and frequency-domain normalization to reduce noise and overfitting, four open-source ByteDance cloud workload datasets, and experiments showing state-of-the-art accuracy, lower compute cost, strong transferability, and practical gains in Kubernetes autoscaling. A plausible implication is that Fremer occupies a specific methodological niche: it is not a general-purpose replacement for all time-series Transformers, but a specialized spectral architecture for cloud workloads whose dominant inductive bias is periodic structure.

Source: https://www.emergentmind.com/topics/fremer