---
title: 'FEATHer: Adaptive Temporal Forecasting Model'
url: https://www.emergentmind.com/topics/fourier-efficient-adaptive-temporal-hierarchy-forecaster-feather
type: topic
---

# FEATHer: Adaptive Temporal Forecasting Model

The Fourier-Efficient Adaptive Temporal Hierarchy Forecaster (FEATHer) is a time-series forecasting architecture designed for accurate long-term predictions on severely resource-constrained edge devices, such as programmable logic controllers and microcontrollers. FEATHer achieves state-of-the-art long-range forecasting accuracy with as few as 400 parameters by combining hand-crafted yet learnable multiscale decomposition, a shared depthwise convolutional temporal mixer, adaptively fused frequency-aware pathway gating, and a sparse, parameter-minimal period extrapolation kernel. Its design is motivated by the need to meet millisecond-level latency and minimal memory usage requirements in industrial and cyber-physical systems, where conventional deep sequence models are often infeasible due to hardware limitations [2601.11350].

## 1. Multiscale Input Decomposition with Frequency Pathways

FEATHer decomposes the input sequence $X \in \mathbb{R}^{L \times D}$ into four parallel, time-aligned branches, each focused on a distinct frequency band:

- **Point branch (b=p):** $k_p=1$
  - $X^{(p)} = \mathrm{DWConv}_{k=1}(\mathrm{InstanceNorm}(X))$
- **High-frequency branch (b=h):** $k_h=3$
  - $X^{(h)} = \mathrm{DWConv}_{k=3}(\mathrm{InstanceNorm}(X))$
- **Mid-frequency branch (b=m):** $k_m=5$
  - $X^{(m)} = \mathrm{DWConv}_{k=5}(\mathrm{InstanceNorm}(X))$
- **Low-frequency branch (b=l):** Downsample then upsample
  - $X^{(l)} = \mathrm{Upsample}_L\left(\mathrm{AvgPool}_r(\mathrm{InstanceNorm}(X))\right)$, $r=4$

Each branch’s kernel is implemented as a depthwise 1D convolution. The low-frequency branch employs stride-based average pooling followed by linear upsampling to filter out high-frequency content with no learnable parameters, resulting in a near-orthogonal filter bank that reduces cross-frequency interference under extreme parameter constraints. All pathway outputs maintain sequence length $L$, easing later fusion.

## 2. Shared Dense Temporal Kernel for Lightweight Temporal Modeling

After multiscale decomposition, each branch is processed by a shared Dense Temporal Kernel (DTK), which performs “projection–depthwise convolution–reverse projection” to maintain parameter efficiency:

- **Linear projection:** $Z^{(b)} = X^{(b)} W_{in}$, $W_{in} \in \mathbb{R}^{D \times S}$
- **Depthwise temporal convolution:** $U^{(b)} = \mathrm{DWConv}_{k_t}(Z^{(b)})$, operates independently on each channel, $k_t$ typically 3 or 5
- **Reverse (output) projection:** $H^{(b)} = U^{(b)} W_{out}$, $W_{out} \in \mathbb{R}^{S \times D}$

Weights $\{W_{in}, W_{out}, \text{DWConv}\}$ are shared across all branches, so total parameter count does not scale with the number of frequency bands. The construction is proven globally Lipschitz with constant at most $\|W_{in}\| \cdot \|K\| \cdot \|W_{out}\|$.

## 3. Frequency-Aware Branch Gating

FEATHer adapts the fusion of branch outputs via instance-dependent, spectrum-driven gating. The gating process is as follows:

1. Normalize $X$ across time.
2. Compute real FFT: $F = \mathrm{FFT}(X_{\text{norm}}) \in \mathbb{C}^{L_f \times D}$
3. Compute magnitude: $A = |F|$
4. Collapse channels: $\vec{a} = \frac{1}{D} \sum_{d=1}^D A_{:,d} \in \mathbb{R}^{L_f}$
5. Gating logits with a 1D ConvNet: $\vec{u} = v(\vec{a}) \in \mathbb{R}^B$
6. Compute softmax weights: $g_b = \exp(u_b) / \sum_{i=1}^B \exp(u_i)$

The fused representation is $H = \sum_{b=1}^B g_b H^{(b)}$, $H \in \mathbb{R}^{L \times D}$, where $B$ is the number of branches. The energy-based gating is derived by entropy-regularized cross-entropy minimization, resulting in a softmax policy $g_b \propto \exp(E_b/\tau)$, where $E_b$ is the relevance (“energy”) of branch $b$.

## 4. Sparse Period Kernel for Forecast Projection

To efficiently map latent states to multistep output forecasts ($H \gg L$), the Sparse Period Kernel (SPK) is used:

- **Sliding residual aggregation:** $H_{agg} = H + \mathrm{DWConv}_{k_{slide}}(H)$, length $L$
- **Phase-alignment:** Choose period $P$ such that $P$ divides $L$ and $H$. For each channel, reshape $H_{agg}$ to $(P, n)$, with $n = L/P$, $m = H/P$
- **Shared linear mapping:** For each phase $p=1…P$, $Y_{p,d} = H_{agg, p, d} W \in \mathbb{R}^{m}$, with $W \in \mathbb{R}^{n \times m}$ shared across channels
- **Recombine:** Interleave all $P$ outputs to form $\hat{Y} \in \mathbb{R}^{H \times D}$

SPK is shown to be parameter-minimal; for phase-aligned period mappings, at least $nm$ parameters are necessary and SPK uses exactly $nm$.

## 5. Parameter Efficiency, Memory, and Computational Characteristics

The FEATHer model can be instantiated with as few as $\sim 400$ trainable parameters:

- **Decomposition filters:** Negligible (fixed, depthwise conv and pooling)
- **Dense Temporal Kernel:** $2 D S + k_t S$ (projection matrices and conv)
- **Sparse Period Kernel:** $n m$

For a typical “ultra-compact” configuration: $D = 1$ (univariate), $S \approx 8$, $k_t=3$, $L=96$, $P=24$, the SPK uses $24 \times (96/24 = 4) = 96$ parameters, and the total is $\lesssim 419$. Peak RAM usage is $< 16$ KB for $D=1$, $L=96$, and $< 32$ KB for moderate $D$. Inference cost is $O(D S + S k_t + P m)$ per step, all terms linear in $L$ and $D$. On ARM Cortex-M3, latency per $96 \rightarrow \{96, 192, 336, 720\}$ forecast varies from $0.08$ ms to $1.4$ ms.

## 6. Experimental Results and Comparative Evaluation

FEATHer was evaluated on eight long-range multivariate time-series benchmarks:

| Dataset         | Granularity      | Application Theme      |
|-----------------|------------------|-----------------------|
| ETTh1, ETTh2    | Hourly           | Electricity demand    |
| AirQuality      | Hourly           | Pollution monitoring  |
| SML             | 15 minutes       | Indoor sensing        |
| Weather         | 10 minutes       | Meteorology           |
| Solar-Energy    | 1 hour           | Solar generation      |
| Traffic         | 1 hour           | Urban traffic         |
| Electricity     | 1 hour           | Household usage       |

Metrics include mean squared error (MSE), mean absolute error (MAE), and Pearson correlation (COR). FEATHer attained the best ranking in 60 cases (across 8 datasets and 4 horizons), with an average rank of $2.05$ versus $3.6$–$4.7$ for the next best baseline. On Solar-Energy, $H=720$ horizon, FEATHer’s MSE was $0.212$ compared to PatchTST’s $0.194$, but FEATHer used $<5$K parameters whereas PatchTST used $>1$M. Ablation studies demonstrated that each core FEATHer component contributed a $2$–$6\%$ improvement in MSE.

## 7. Deployment Implications for Edge Inference

FEATHer’s architectural minimalism enables real-time deployment on microcontrollers and PLCs. On ARM Cortex-M3, inference latency (for $L=96, H=96$) is $\approx 0.08$ ms, rising to $0.6$–$1.4$ ms for $H=720$. Peak RAM usage is $< 16$ KB for single-channel input, and $< 32$ KB for eight-channel cases. For comparison, transformer models require $50$–$500$ KB activations and $>100$ ms latency. This resource profile satisfies stringent requirements in industrial automation, embedded control, and energy-constrained settings, where on-chip SRAM and sub-millisecond latency are indispensable [2601.11350]. A plausible implication is that FEATHer’s approach provides a template for scalable, data-efficient forecasting in safety-critical cyber-physical systems operating with on-chip resources.

Source: https://www.emergentmind.com/topics/fourier-efficient-adaptive-temporal-hierarchy-forecaster-feather