---
title: Logsparse Decomposable Multiscaling (LDM)
url: https://www.emergentmind.com/topics/logsparse-decomposable-multiscaling-ldm
type: topic
---

# Logsparse Decomposable Multiscaling (LDM)

Logsparse Decomposable Multiscaling (LDM) is a framework designed to address the challenges of long-context modeling in long-term time-series forecasting (LTSF). LDM combines multiscale decomposition with logsparse truncation to achieve efficient and effective forecasting over long horizons, particularly by reducing non-stationarity and alleviating both memory and computational bottlenecks. Through explicit task separation by scale and parallelized per-scale processing, LDM outperforms state-of-the-art forecasting models on standard benchmarks while using significantly fewer computational resources [2412.16572].

## 1. Formal Structure and Mathematical Definition

LDM applies a modified wavelet-like filter bank for multiscale decomposition of the input time series. Given a multivariate sequence 
$s^0(k) \in \mathbb{R}^M$ for $k=1,\dots,L$, with $M$ the channel dimension and $L$ the sequence length, and a strictly increasing set of scales $\{p^1<p^2<\cdots<p^N\}$, the framework recursively performs:

- **Low-pass filtering:** 
  \[
  \tau^i(k) = \frac{1}{p^i} \sum_{n=-\frac{p^i}{2}}^{\frac{p^i}{2}} s^i(k-n)
  \]
- **Downsampling and detail extraction:** 
  \[
  s^{i+1}(k) = \downarrow_{\frac{p^i}{2}} \tau^i(k), \qquad w^i(k) = s^i(k) - \tau^i(k)
  \]
  where $\downarrow_q$ denotes average pooling with kernel and stride $q$. After $N$ iterations, the decomposition yields 
  $\{w^0,w^1,...,w^{N-1},s^N\}$, representing increasingly coarser scales.

- **Logsparse truncation:** Each component $s_n$ or $w_n$ is truncated to
  \[
  \tilde L_n = \min(L_n, \; \tfrac{p^n}{\eta})
  \]
  where $L_n \approx 2L/p^n$ is the length after downsampling and $\eta \in (0,1]$ is the sparsity parameter regulating truncation severity.

- **Scale-wise prediction and fusion:** Each truncated component $\mathbf{s}_n \in \mathbb{R}^{M \times \tilde{L}_n}$ is input to its own predictor, $\mathrm{Pred}_n$, yielding $\mathbf{y}_n \in \mathbb{R}^{M \times H_n}$ with $H_n \approx 2H/p^n$. The final forecast is
  \[
  \hat{\mathbf{y}} = \sum_{n=1}^{N+1} \mathrm{Interpolate}_{H_n\rightarrow H}(\mathbf{y}_n)
  \]
  where upsampling to the horizon $H$ is achieved by linear interpolation.

## 2. Multiscale Decoupling and Non-Stationarity Reduction

The LDM framework explicitly decouples the input time series across frequency scales:

- **Finest scales ($w^0$, $w^1$):** Represent high-frequency detail components that, after local mean subtraction, are approximately stationary.
- **Coarsest scale ($s^N$):** Captures the long-term trend, typically smooth and potentially non-stationary, but represented with very low dimension (short length after downsampling).
- **Task assignment:** By decomposing trend and detail, each predictor is optimized on the scale most appropriate for its temporal structure, which results in improved predictability due to reduced non-stationarity within each subtask.

This scale-wise task separation both enhances statistical stationarity in detail components and enables architectural simplification by obviating costly cross-scale interactions.

## 3. Architectural Design and Processing Pipeline

LDM’s end-to-end forecasting pipeline comprises four main stages:

- **Decomposition:** The input is subjected to multiscale filtering and downsampling, extracting components at each scale as described above.
- **Logsparse truncation:** Efficiently truncates each scale’s input to retain only the recent $\propto 1/\eta$ fraction, eliminating long, noisy tails particularly at fine scales.
- **Parallelized per-scale prediction:** Each truncated scale component is input to its own, typically compact, predictor block (e.g., small-scale Transformer or MLP). There is no explicit cross-scale attention or gating mechanism; the architecture is fully parallel across scales.
- **Fusion:** All per-scale outputs $\mathbf{y}_n$ are linearly interpolated to the full horizon and summed element-wise to form the final forecast $\hat{\mathbf{y}}$.

This pipeline eliminates architectural complexity while ensuring each model subcomponent operates on appropriately tailored input.

## 4. Training Objective and Loss Function

The primary loss function is the mean squared error (MSE) between the final forecast and the ground truth:
\[
\mathcal{L} = \frac{1}{T} \sum_{t=1}^T \frac{1}{MH} \sum_{i=1}^M \sum_{j=1}^H \bigl[y_t^{(i,j)} - \hat y_t^{(i,j)}\bigr]^2
\]
where $T$ is batch size, $M$ channel count, $H$ the forecasting horizon, $y_t$ the ground truth, and $\hat{y}_t$ the LDM prediction. Although scale-wise auxiliary losses $\mathcal{L}_n$ can be incorporated, by default the loss is computed only over the fused output.

## 5. Computational Efficiency and Scalability

LDM’s efficiency arises fundamentally from its logsparse representation:

- **Effective context reduction:** Truncating each scale to $\tilde L_n \ll L_n$ means that each predictor operates on significantly reduced-length sequences, with $\tilde L_1 \approx p^1/\eta \ll L$ even for the largest predictor.
- **Complexity analysis:** While a standard Transformer has complexity $O(L^2)$ for input length $L$, and segment/token-based variants $O(M L_{\mathrm{seg}}^2)$ for segment length $L_{\mathrm{seg}}$, LDM reduces this to
  \[
  O\left(M\,\left(\frac{\tilde{L}_1}{L_{\mathrm{seg}}}\right)^2\right)
  \]
  with $\tilde{L}_1$ observed to be 5–10× smaller than $L$, leading to proportionate savings in runtime and memory.

## 6. Empirical Performance on Long-Term Forecasting Benchmarks

LDM has been evaluated on eight standard multivariate LTSF datasets (Weather, Solar-Energy, Electricity, Traffic, ETTh1, ETTh2, ETTm1, ETTm2) with forecasting horizons $H\in\{96,192,336,720\}$. Performance is measured by MSE and MAE, with key results summarized below (values averaged over horizons):

| Dataset      | LDM MSE | LDM MAE | PatchTST MSE | PatchTST MAE |
|--------------|---------|---------|--------------|--------------|
| Weather      | 0.222   | 0.267   | 0.237        | 0.280        |
| Solar-Energy | 0.188   | 0.266   | 0.205        | 0.280        |
| Electricity  | 0.183   | 0.254   | 0.197        | 0.267        |
| Traffic      | 0.406   | 0.272   | 0.415        | 0.286        |
| ETTh1        | 0.404   | 0.438   | 0.438        | 0.449        |
| ETTh2        | 0.344   | 0.380   | 0.404        | 0.417        |
| ETTm1        | 0.346   | 0.375   | 0.386        | 0.395        |
| ETTm2        | 0.248   | 0.313   | 0.280        | 0.333        |

Across all datasets, LDM reduces MSE by 3–15% versus the strongest Transformer baseline (PatchTST) and by 10–20% against the best MLP baseline (TimeMixer) while using 30–50% less GPU memory and 20–40% less training time on large $L$.

## 7. Summary and Implications

LDM offers a framework that decomposes time series into a parsimonious number of scale-wise streams, applies log-sparse truncation to each, processes scale-wise inputs independently—avoiding the complexity of cross-scale attention—and fuses predictions by summation after appropriate interpolation. These design choices enable both superior forecasting accuracy and substantial resource savings on long-sequence forecasting tasks, directly addressing overfitting and context bottlenecks pervasive in deep model architectures for LTSF [2412.16572]. A plausible implication is that similar logsparse decomposable approaches may prove beneficial in other long-sequence modeling domains where non-stationarity and context length pose practical constraints.

Source: https://www.emergentmind.com/topics/logsparse-decomposable-multiscaling-ldm