Local Trend-Aware Attention

• Local Trend-Aware Attention is a mechanism that replaces global token aggregation with locally contextualized and trend-sensitive operations using causal convolutions and hierarchical local time boxes.
• It enhances the capture of short-term dynamics such as turning tendency, deceleration, and lane-following curvature by focusing on immediate temporal changes.
• Empirical evaluations demonstrate improved trajectory forecasting and reduced computational complexity by limiting interactions to relevant local neighborhoods.

Local Trend-Aware Attention denotes a family of attention mechanisms that replace indiscriminate global token–token aggregation with locality-constrained, direction-sensitive, or locally contextualized matching so that short-range evolution influences the attention weights. The term is used explicitly in LTMSformer for multi-agent trajectory prediction, where Local Trend-Aware Attention (LTAA) combines causal convolution with hierarchical local time boxes; closely related formulations appear as local structure-aware spiking self-attention, local spectral attention, multiscale trend-aware self-attention, and trend-aware gated modulation in spiking vision, speech enhancement, financial forecasting, and long-term time-series forecasting (Yan et al., 7 Jul 2025, Li et al., 12 May 2026, Hou et al., 2023, Liu et al., 14 Dec 2025, Liu et al., 26 Jun 2026). This suggests a design principle rather than a single canonical operator: attention is made sensitive to local continuity, local drift, or local structural compatibility before or during aggregation.

1. Conceptual scope and motivation

The central motivation is that standard self-attention often treats the entire sequence or feature map as a homogeneous interaction space, whereas many prediction problems are governed by strong short-range structure. In LTMSformer, the motivating examples are recent turning tendency, deceleration, lane-following curvature, and the onset of an evasive maneuver; the paper argues that the state at time $t$ is typically most strongly correlated with nearby times $t-1,t-2,\dots$, and that full-sequence aggregation can weaken this bias toward adjacent timesteps (Yan et al., 7 Jul 2025). In EXFormer, the analogous claim is that vanilla self-attention performs pointwise matching and can align equal-valued but contextually different observations, whereas local-window representations can align observations on the basis of local slopes and remain sensitive to regime shifts (Liu et al., 14 Dec 2025).

Across the literature, “local” and “trend-aware” are not used uniformly. In LTMSformer and EXFormer, trend-awareness is attached to temporal motion or return dynamics. In LSFormer, the comparable idea is local structure awareness: horizontal and vertical dilated windows capture directional structural patterns in spatial features rather than temporal slopes (Li et al., 12 May 2026). In TA-SparseMG, trend-awareness is implemented through local drift statistics, multiscale smoothing, and gated residual suppression rather than explicit query–key–value attention (Liu et al., 26 Jun 2026). In speech enhancement, local spectral attention restricts each frequency band to nearby bands, thereby favoring local spectral continuity and short-range correlation rather than explicit trend estimation (Hou et al., 2023). A common misconception is therefore that local trend-aware attention must mean explicit slope computation inside a Transformer; several papers instead realize the same inductive bias through masks, convolutions, local statistics, gates, or learned affinity functions.

A second recurring motivation is computational. Global attention induces quadratic interaction graphs, while many papers argue that only a structured subset of those interactions is task-relevant. LSFormer explicitly frames global self-attention in spiking models as computationally redundant and mismatched to spike sparsity, and local spectral attention similarly argues that full-range frequency attention can introduce weakly related interactions and residual noise in full-band speech enhancement (Li et al., 12 May 2026, Hou et al., 2023).

2. LTAA in LTMSformer

In LTMSformer, LTAA is the temporal module inside the first-stage local temporal-spatial encoder. It appears after the Agent-Agent Encoder and before the Motion State Encoder (MSE). The overall pipeline is: agent-centric, translation-invariant, rotation-aligned preprocessing; agent–agent interaction encoding; LTAA for local temporal trend extraction; MSE for high-order motion-state-enhanced spatial interaction; Agent-Lane Encoder; Global Interaction block; multimodal decoder; and Lightweight Proposal Refinement Module (Yan et al., 7 Jul 2025).

The agent-centric temporal sequence is constructed from motion increments. For central agent $i$ at time $t$,

$$\Delta \mathbf{p}_i^t = \mathbf{p}_i^t - \mathbf{p}_i^{t-1}, \qquad \Delta \mathbf{p}_{ij}^t = \Delta \mathbf{p}_j^t - \Delta \mathbf{p}_i^t.$$

Using the rotation matrix $\mathbf{R}_i^{T_o}$ defined by the central agent’s current heading at the final observation step $T_o$,

$$\mathbf{h}_i^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_i^t, \qquad \mathbf{h}_{ij}^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_{ij}^t .$$

The Agent-Agent Encoder embeds these with MLPs and applies multi-head attention,

$$\mathbf{s}_i^t = \phi_{\text{center}}(\mathbf{h}_i^t), \qquad \mathbf{s}_{ij}^t = \phi_{\text{nbr}}([\mathbf{h}_j^t, \mathbf{h}_{ij}^t]),$$

$$\mathbf{c}_i^t = \mathrm{MHA}(\mathbf{s}_i^t, \mathbf{s}_{ij}^t), \qquad \mathbf{z}_i = \{\mathbf{c}_i^t\}_{t=1}^{T_o}, \quad \mathbf{z}_i \in \mathbb{R}^{T_o \times d_h}.$$

As in HiVT, a learnable token $t-1,t-2,\dots$0 is appended and learnable positional embeddings are added, producing $t-1,t-2,\dots$1. The paper notes a minor notation inconsistency because subsequent equations use $t-1,t-2,\dots$2 as the convolution input while the textual description indicates that the token-augmented sequence is fed into LTAA. This suggests that the intended input is the temporally ordered token sequence $t-1,t-2,\dots$3 (Yan et al., 7 Jul 2025).

LTAA does not form $t-1,t-2,\dots$4 and $t-1,t-2,\dots$5 by direct linear projections. Instead, it applies a causal temporal convolution with kernel size $t-1,t-2,\dots$6, followed by BatchNorm, to generate trend-aware queries and keys: $t-1,t-2,\dots$7 The convolution is causal, so the representation at time $t-1,t-2,\dots$8 depends only on the current and previous $t-1,t-2,\dots$9 tokens. The paper describes this as a “convolutional attention mechanism”: local temporal filtering shapes the attention inputs before attention is applied. LTAA then performs local box convolutional attention, denoted LMHA, inside non-overlapping local time boxes,

$i$0

with three hierarchical layers and box sizes

$i$1

Thus the lower layers attend within short temporal neighborhoods, while the upper layer effectively spans nearly the full 2-second, 10 Hz Argoverse 1 observation history. The outputs are then combined as

$i$2

although the exact operator $i$3 is not defined. The feedforward sublayer uses GELU, with the paper stating: “Similar to BERT, we use the GELU activation function in the feedforward layer.”

The interface to spatial reasoning is explicit. MSE constructs neighbor motion-state vectors

$i$4

where the attributes include acceleration, jerk, and heading. An MLP $i$5 embeds these features, and LTAA’s output is used as the query: $i$6 This division of labor is architecturally central: LTAA models the central agent’s interaction-aware temporal sequence, whereas MSE uses that temporally informed representation to decide how to attend to neighbors’ high-order motion attributes (Yan et al., 7 Jul 2025).

3. Recurrent mechanism families

Representative mechanisms can be organized by how locality is enforced and how “trend-awareness” is realized.

Representative paper	Locality mechanism	Meaning of trend-awareness
LTMSformer (Yan et al., 7 Jul 2025)	Causal convolution for $i$7 and non-overlapping local time boxes $i$8	Short-term motion trends such as turning tendency, deceleration, and lane-following curvature
EXFormer (Liu et al., 14 Dec 2025)	Parallel convolutional branches for $i$9 with kernel sizes such as $t$0	Alignment on the basis of local slopes and regime-shift sensitivity
LSFormer (Li et al., 12 May 2026)	Overlapping horizontal and vertical local windows with dilation rates $t$1	Directional structural patterns and multi-scale local continuity
TA-SparseMG (Liu et al., 26 Jun 2026)	Multiscale smoothing, depthwise-convolutional gates, and gated-attention MLP	Local drift statistics, denoising around local trend, and adaptive feature modulation
Local spectral attention (Hou et al., 2023)	Frequency mask $t$2	Nearby spectral correlation and local spectral continuity
HA-GAT and CHGAT (Wang et al., 2023, Zhu et al., 2021)	Local distribution exploration or context-conditioned graph attention	Local neighborhood distributions or real-time local context as the conditioning signal

One family replaces pointwise query–key projections with locally contextualized projections. LTAA and EXFormer are the clearest cases. LTAA uses causal temporal convolution before attention, while EXFormer replaces the linear projections for $t$3 and $t$4 with parallel Conv2D branches over the temporal axis, leaving $t$5 as a shared linear projection and then fusing branch-specific attention outputs (Yan et al., 7 Jul 2025, Liu et al., 14 Dec 2025).

A second family restricts the interaction graph itself. Local spectral attention introduces an additive mask

$t$6

so that each spectral band attends only within a finite frequency neighborhood. LSFormer applies local sparse attention through directional 1D windows sampled along horizontal and vertical axes rather than through non-overlapping 2D windows; channel groups receive different dilation rates, yielding multi-scale receptive fields without changing the asymptotic complexity (Hou et al., 2023, Li et al., 12 May 2026).

A third family realizes local trend awareness without classical dot-product attention. TA-SparseMG computes local drift from consecutive subsegments,

$t$7

uses multiscale smoothing and residual gating,

$t$8

and applies a gated-attention MLP head rather than self-attention (Liu et al., 26 Jun 2026). Segmentation-aware convolutional networks use learned local attention masks

$t$9

which gate local aggregation so that only neighbors estimated to belong to the same region contribute strongly (Harley et al., 2017). HA-GAT similarly converts local node distributions into edge-type-aware weights through

$$\Delta \mathbf{p}_i^t = \mathbf{p}_i^t - \mathbf{p}_i^{t-1}, \qquad \Delta \mathbf{p}_{ij}^t = \Delta \mathbf{p}_j^t - \Delta \mathbf{p}_i^t.$$0

while CHGAT makes the score of a historical graph vertex depend on the current outside context through type-specific MLPs (Wang et al., 2023, Zhu et al., 2021).

4. Cross-domain realizations

In long-term time-series forecasting, TA-SparseMG is notable because it explicitly denies the need for full Transformer self-attention. Trend-awareness appears at three levels: distribution calibration through trend-aware reversible instance normalization, feature purification through scale-adaptive gated denoising, and prediction-level emphasis through multiscale gated-attention in the forecasting head. The paper states that its “attention” is a learned sigmoid importance map over hidden responses rather than $$\Delta \mathbf{p}_i^t = \mathbf{p}_i^t - \mathbf{p}_i^{t-1}, \qquad \Delta \mathbf{p}_{ij}^t = \Delta \mathbf{p}_j^t - \Delta \mathbf{p}_i^t.$$1, so local trend-aware attention is realized as selective modulation rather than pairwise token affinity (Liu et al., 26 Jun 2026).

In financial forecasting, EXFormer supplies a more orthodox Transformer variant. For each branch $$\Delta \mathbf{p}_i^t = \mathbf{p}_i^t - \mathbf{p}_i^{t-1}, \qquad \Delta \mathbf{p}_{ij}^t = \Delta \mathbf{p}_j^t - \Delta \mathbf{p}_i^t.$$2,

$$\Delta \mathbf{p}_i^t = \mathbf{p}_i^t - \mathbf{p}_i^{t-1}, \qquad \Delta \mathbf{p}_{ij}^t = \Delta \mathbf{p}_j^t - \Delta \mathbf{p}_i^t.$$3

and branch-specific attention is

$$\Delta \mathbf{p}_i^t = \mathbf{p}_i^t - \mathbf{p}_i^{t-1}, \qquad \Delta \mathbf{p}_{ij}^t = \Delta \mathbf{p}_j^t - \Delta \mathbf{p}_i^t.$$4

followed by fusion

$$\Delta \mathbf{p}_i^t = \mathbf{p}_i^t - \mathbf{p}_i^{t-1}, \qquad \Delta \mathbf{p}_{ij}^t = \Delta \mathbf{p}_j^t - \Delta \mathbf{p}_i^t.$$5

The paper explicitly describes this as alignment “on the basis of local slopes,” with three branches intended to capture short-, medium-, and long-horizon dynamics (Liu et al., 14 Dec 2025).

In spiking vision, LSFormer’s Local Structure-Aware Spiking Self-Attention is not temporal trend modeling, but it is a close analogue in spatial form. Queries, keys, and values are produced by conv-BN-spiking layers, channels are split into groups with different dilation rates, and each query location attends to short horizontal and vertical line neighborhoods,

$$\Delta \mathbf{p}_i^t = \mathbf{p}_i^t - \mathbf{p}_i^{t-1}, \qquad \Delta \mathbf{p}_{ij}^t = \Delta \mathbf{p}_j^t - \Delta \mathbf{p}_i^t.$$6

This yields a directional, multi-scale, locality-preserving attention mechanism tailored to sparse spike activity (Li et al., 12 May 2026).

In speech enhancement, local spectral attention is a particularly clean construction because it changes only the support of frequency-wise self-attention. The localized spectral attention output

$$\Delta \mathbf{p}_i^t = \mathbf{p}_i^t - \mathbf{p}_i^{t-1}, \qquad \Delta \mathbf{p}_{ij}^t = \Delta \mathbf{p}_j^t - \Delta \mathbf{p}_i^t.$$7

ensures that each frequency bin attends only to nearby bins. The paper argues that full-range spectral attention can be harmful in full-band speech enhancement because low- and high-frequency regions may exhibit weak similarity and induce excessive residual noise (Hou et al., 2023).

Graph- and context-based variants show that locality can also be semantic or contextual rather than geometric. HA-GAT treats local distribution as the underlying heterophily signal through a learned explorer network,

$$\Delta \mathbf{p}_i^t = \mathbf{p}_i^t - \mathbf{p}_i^{t-1}, \qquad \Delta \mathbf{p}_{ij}^t = \Delta \mathbf{p}_j^t - \Delta \mathbf{p}_i^t.$$8

then uses $$\Delta \mathbf{p}_i^t = \mathbf{p}_i^t - \mathbf{p}_i^{t-1}, \qquad \Delta \mathbf{p}_{ij}^t = \Delta \mathbf{p}_j^t - \Delta \mathbf{p}_i^t.$$9 to parameterize edge-type-aware attention (Wang et al., 2023). CHGAT, designed for local consumer service platforms, makes vertex- and path-level attention explicitly context-conditioned: $\mathbf{R}_i^{T_o}$0 so current query, candidate item, and real-time context influence which historical nodes and meta-paths are aggregated (Zhu et al., 2021).

5. Empirical behavior and computational trade-offs

The empirical record in the supplied literature is broadly favorable but heterogeneous. On Argoverse 1, the full LTMSformer improves over HiVT-64 by reducing validation minADE from $\mathbf{R}_i^{T_o}$1 to $\mathbf{R}_i^{T_o}$2, minFDE from $\mathbf{R}_i^{T_o}$3 to $\mathbf{R}_i^{T_o}$4, and MR from $\mathbf{R}_i^{T_o}$5 to $\mathbf{R}_i^{T_o}$6; on the test set it reduces minADE from $\mathbf{R}_i^{T_o}$7 to $\mathbf{R}_i^{T_o}$8, minFDE from $\mathbf{R}_i^{T_o}$9 to $T_o$0, and MR from $T_o$1 to $T_o$2. The isolated LTAA contribution in the ablation is positive but modest: after adding LTAA to the MSE-enhanced variant, minADE improves from $T_o$3 to $T_o$4, minFDE from $T_o$5 to $T_o$6, and MR from $T_o$7 to $T_o$8. The paper’s qualitative analysis nevertheless states that LTAA keeps predicted trajectories within lane boundaries and aligns trajectory trends more closely with ground truth (Yan et al., 7 Jul 2025).

LSFormer reports stronger isolated gains for locality-aware spatial attention in spiking vision. Replacing SSA with LS-SSA yields $T_o$9 on CIFAR10-DVS and $$\mathbf{h}_i^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_i^t, \qquad \mathbf{h}_{ij}^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_{ij}^t .$$0 on CIFAR-100, while the full model reaches $$\mathbf{h}_i^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_i^t, \qquad \mathbf{h}_{ij}^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_{ij}^t .$$1 on Tiny-ImageNet and $$\mathbf{h}_i^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_i^t, \qquad \mathbf{h}_{ij}^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_{ij}^t .$$2 on N-CALTECH101. Complexity is reduced from $$\mathbf{h}_i^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_i^t, \qquad \mathbf{h}_{ij}^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_{ij}^t .$$3 and $$\mathbf{h}_i^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_i^t, \qquad \mathbf{h}_{ij}^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_{ij}^t .$$4 for global attention to $$\mathbf{h}_i^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_i^t, \qquad \mathbf{h}_{ij}^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_{ij}^t .$$5 and $$\mathbf{h}_i^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_i^t, \qquad \mathbf{h}_{ij}^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_{ij}^t .$$6 for LS-SSA (Li et al., 12 May 2026).

TA-SparseMG reports that, across six long-term time-series forecasting benchmarks with $$\mathbf{h}_i^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_i^t, \qquad \mathbf{h}_{ij}^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_{ij}^t .$$7 and $$\mathbf{h}_i^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_i^t, \qquad \mathbf{h}_{ij}^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_{ij}^t .$$8, it improves over SparseTSF/MLP in $$\mathbf{h}_i^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_i^t, \qquad \mathbf{h}_{ij}^{t} = \mathbf{R}_i^{T_o} \Delta \mathbf{p}_{ij}^t .$$9 settings on MSE and $$\mathbf{s}_i^t = \phi_{\text{center}}(\mathbf{h}_i^t), \qquad \mathbf{s}_{ij}^t = \phi_{\text{nbr}}([\mathbf{h}_j^t, \mathbf{h}_{ij}^t]),$$0 on MAE. The average MSE increases from $$\mathbf{s}_i^t = \phi_{\text{center}}(\mathbf{h}_i^t), \qquad \mathbf{s}_{ij}^t = \phi_{\text{nbr}}([\mathbf{h}_j^t, \mathbf{h}_{ij}^t]),$$1 in the full model to $$\mathbf{s}_i^t = \phi_{\text{center}}(\mathbf{h}_i^t), \qquad \mathbf{s}_{ij}^t = \phi_{\text{nbr}}([\mathbf{h}_j^t, \mathbf{h}_{ij}^t]),$$2 without TA-RevIN, $$\mathbf{s}_i^t = \phi_{\text{center}}(\mathbf{h}_i^t), \qquad \mathbf{s}_{ij}^t = \phi_{\text{nbr}}([\mathbf{h}_j^t, \mathbf{h}_{ij}^t]),$$3 without denoising, and $$\mathbf{s}_i^t = \phi_{\text{center}}(\mathbf{h}_i^t), \qquad \mathbf{s}_{ij}^t = \phi_{\text{nbr}}([\mathbf{h}_j^t, \mathbf{h}_{ij}^t]),$$4 without MSGA, indicating that all three local trend-aware components contribute (Liu et al., 26 Jun 2026).

Local spectral attention improves both MTFAA and DPARN on full-band VoiceBank+DEMAND. In causal MTFAA, PESQ rises from $$\mathbf{s}_i^t = \phi_{\text{center}}(\mathbf{h}_i^t), \qquad \mathbf{s}_{ij}^t = \phi_{\text{nbr}}([\mathbf{h}_j^t, \mathbf{h}_{ij}^t]),$$5 to $$\mathbf{s}_i^t = \phi_{\text{center}}(\mathbf{h}_i^t), \qquad \mathbf{s}_{ij}^t = \phi_{\text{nbr}}([\mathbf{h}_j^t, \mathbf{h}_{ij}^t]),$$6, CBAK from $$\mathbf{s}_i^t = \phi_{\text{center}}(\mathbf{h}_i^t), \qquad \mathbf{s}_{ij}^t = \phi_{\text{nbr}}([\mathbf{h}_j^t, \mathbf{h}_{ij}^t]),$$7 to $$\mathbf{s}_i^t = \phi_{\text{center}}(\mathbf{h}_i^t), \qquad \mathbf{s}_{ij}^t = \phi_{\text{nbr}}([\mathbf{h}_j^t, \mathbf{h}_{ij}^t]),$$8, and SiSDR from $$\mathbf{s}_i^t = \phi_{\text{center}}(\mathbf{h}_i^t), \qquad \mathbf{s}_{ij}^t = \phi_{\text{nbr}}([\mathbf{h}_j^t, \mathbf{h}_{ij}^t]),$$9 dB to $$\mathbf{c}_i^t = \mathrm{MHA}(\mathbf{s}_i^t, \mathbf{s}_{ij}^t), \qquad \mathbf{z}_i = \{\mathbf{c}_i^t\}_{t=1}^{T_o}, \quad \mathbf{z}_i \in \mathbb{R}^{T_o \times d_h}.$$0 dB after replacing global frequency attention with local spectral attention; analogous improvements are reported for causal DPARN (Hou et al., 2023).

EXFormer reports that replacing its multi-scale trend-aware attention with traditional self-attention is especially detrimental for USD/JPY, with the full model’s advantage peaking at a $$\mathbf{c}_i^t = \mathrm{MHA}(\mathbf{s}_i^t, \mathbf{s}_{ij}^t), \qquad \mathbf{z}_i = \{\mathbf{c}_i^t\}_{t=1}^{T_o}, \quad \mathbf{z}_i \in \mathbb{R}^{T_o \times d_h}.$$1 improvement at $$\mathbf{c}_i^t = \mathrm{MHA}(\mathbf{s}_i^t, \mathbf{s}_{ij}^t), \qquad \mathbf{z}_i = \{\mathbf{c}_i^t\}_{t=1}^{T_o}, \quad \mathbf{z}_i \in \mathbb{R}^{T_o \times d_h}.$$2 and remaining $$\mathbf{c}_i^t = \mathrm{MHA}(\mathbf{s}_i^t, \mathbf{s}_{ij}^t), \qquad \mathbf{z}_i = \{\mathbf{c}_i^t\}_{t=1}^{T_o}, \quad \mathbf{z}_i \in \mathbb{R}^{T_o \times d_h}.$$3 at $$\mathbf{c}_i^t = \mathrm{MHA}(\mathbf{s}_i^t, \mathbf{s}_{ij}^t), \qquad \mathbf{z}_i = \{\mathbf{c}_i^t\}_{t=1}^{T_o}, \quad \mathbf{z}_i \in \mathbb{R}^{T_o \times d_h}.$$4. More broadly, it reports directional-accuracy gains of up to $$\mathbf{c}_i^t = \mathrm{MHA}(\mathbf{s}_i^t, \mathbf{s}_{ij}^t), \qquad \mathbf{z}_i = \{\mathbf{c}_i^t\}_{t=1}^{T_o}, \quad \mathbf{z}_i \in \mathbb{R}^{T_o \times d_h}.$$5–$$\mathbf{c}_i^t = \mathrm{MHA}(\mathbf{s}_i^t, \mathbf{s}_{ij}^t), \qquad \mathbf{z}_i = \{\mathbf{c}_i^t\}_{t=1}^{T_o}, \quad \mathbf{z}_i \in \mathbb{R}^{T_o \times d_h}.$$6 over baselines and cumulative trading returns of $$\mathbf{c}_i^t = \mathrm{MHA}(\mathbf{s}_i^t, \mathbf{s}_{ij}^t), \qquad \mathbf{z}_i = \{\mathbf{c}_i^t\}_{t=1}^{T_o}, \quad \mathbf{z}_i \in \mathbb{R}^{T_o \times d_h}.$$7, $$\mathbf{c}_i^t = \mathrm{MHA}(\mathbf{s}_i^t, \mathbf{s}_{ij}^t), \qquad \mathbf{z}_i = \{\mathbf{c}_i^t\}_{t=1}^{T_o}, \quad \mathbf{z}_i \in \mathbb{R}^{T_o \times d_h}.$$8, and $$\mathbf{c}_i^t = \mathrm{MHA}(\mathbf{s}_i^t, \mathbf{s}_{ij}^t), \qquad \mathbf{z}_i = \{\mathbf{c}_i^t\}_{t=1}^{T_o}, \quad \mathbf{z}_i \in \mathbb{R}^{T_o \times d_h}.$$9 before conservative transaction costs and slippage are accounted for (Liu et al., 14 Dec 2025).

From a computational standpoint, the trade-off is consistent across domains: locality usually reduces interaction complexity, but not always total model cost. LTAA changes the dense temporal interaction graph into box-wise attention, which the paper interprets as lower “parameter complexity,” although the clearest direct effect is on attention computation, from full-sequence $t-1,t-2,\dots$00 behavior to approximately $t-1,t-2,\dots$01 for box size $t-1,t-2,\dots$02 (Yan et al., 7 Jul 2025). LS-SSA similarly lowers asymptotic attention complexity, whereas EXFormer preserves dense temporal attention and instead adds convolutional preprocessing branches. A plausible implication is that “local trend-aware” should not be conflated with “lightweight”; some designs are primarily inductive-bias modifications rather than strict efficiency devices.

6. Limitations, ambiguities, and open questions

The literature leaves several technical points under-specified. LTAA is only partially formalized: the exact LMHA formula, the $t-1,t-2,\dots$03 operator, edge handling for non-overlapping boxes, and any residual or normalization details inside LTAA are omitted. The paper itself notes a notation inconsistency regarding whether the token-augmented sequence $t-1,t-2,\dots$04 or the stacked feature matrix $t-1,t-2,\dots$05 is the actual convolution input (Yan et al., 7 Jul 2025). EXFormer’s trend-aware attention is clearer conceptually than typographically; the manuscript gives the branch construction but does not fully specify head-wise tensor organization or grouped-convolution details (Liu et al., 14 Dec 2025).

A recurrent conceptual ambiguity is the meaning of “trend-aware.” In TA-SparseMG, there is no classical local attention window over tokens; trend-awareness is statistical and gate-based (Liu et al., 26 Jun 2026). In LSFormer and LoGoCAF, the analogous mechanisms are directional and structural rather than temporal (Li et al., 12 May 2026, Zhang et al., 2024). In topology-aware attention for forecasting, the local term is explicitly geometric and topological rather than based on slope or drift, using local subwindows, persistent-homology summaries, and a validation-gated local residual (Faghihi et al., 4 May 2026). Thus, the phrase spans explicit local temporal motion trends, local spectral continuity, local structural organization, local distributional heterophily, and local topological change.

Another open issue is boundary behavior. Non-overlapping local boxes in LTAA may create transition artifacts unless compensated by hierarchical scales, but the paper does not discuss this (Yan et al., 7 Jul 2025). Local spectral attention fixes the span $t-1,t-2,\dots$06 manually and does not adapt the neighborhood to signal content (Hou et al., 2023). LSFormer’s dilation study shows that excessively large dilation can exceed effective feature-map size and become overly sparse or noisy, indicating that local trend-aware mechanisms depend critically on neighborhood design remaining aligned with the underlying structure (Li et al., 12 May 2026).

A final misconception is that local trend-aware attention necessarily supersedes global modeling. The surveyed papers more often combine local and global reasoning than replace one with the other. LTAA expands receptive field hierarchically from box sizes $t-1,t-2,\dots$07 to $t-1,t-2,\dots$08; LSFormer combines local directional windows with multi-dilation grouping; LoGoCAF uses convolutions in shallow stages and transformers in deeper stages; topology-aware forecasting adds a validation-gated local residual to a global topology-biased attention backbone (Yan et al., 7 Jul 2025, Li et al., 12 May 2026, Zhang et al., 2024, Faghihi et al., 4 May 2026). The dominant pattern is therefore not localism alone, but a structured rebalancing of local and global dependencies so that adjacent evolution, neighborhood structure, or contextual drift is not erased by globally uniform attention.