---
title: Semantic-aware AFT for Stock Prediction
url: https://www.emergentmind.com/topics/semantic-aware-aft-saft
type: topic
---

# Semantic-aware AFT for Stock Prediction

Semantic-aware AFT (SAFT) is a Transformer-based architecture for stock return prediction under non-stationary financial markets. In the formulation described in "Adaptive Financial Transformer with Regime-Gated Attention for Stock Return Prediction," SAFT extends a standard Transformer encoder with a Market Regime Encoder, an Adaptive Gate Network, and an Adaptive Financial Context module, allowing self-attention to be dynamically biased by semantic relationships between financial indicators rather than treating all input features uniformly. The model groups 95 engineered financial features into 11 semantic categories, uses corrected sequence alignment and non-overlapping backtesting, and optimizes a financially-aware composite objective that combines prediction error, directional accuracy, and non-overlapping Sharpe ratio [2606.29347].

## 1. Architectural formulation

SAFT is defined as an extension of the standard Transformer encoder with three domain-specific modules: a Market Regime Encoder that compresses group-wise feature statistics into a latent regime vector, an Adaptive Gate Network that turns the regime vector into per-group gate weights, and an Adaptive Financial Context that uses those weights to bias the self-attention scores via group-wise semantic similarity [2606.29347].

The input is a batch of lookback windows
$$
X \in \mathbb{R}^{B \times L \times D}
$$
with $D=95$ technical features over $L=60$ trading days. The sequence is projected to
$$
E = XW_{\mathrm{embed}} + b_{\mathrm{embed}} \in \mathbb{R}^{B \times L \times d_{\mathrm{model}}},
$$
prepended with a learnable CLS token, and added to sinusoidal positional encoding. The Market Regime Encoder operates on each semantic feature group $X_g$ and computes
$$
h_g = \mathrm{Pool}\bigl(\mathrm{GELU}(X_g W_g + b_g)\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}},
$$
followed by concatenation and fusion:
$$
R = \mathrm{GELU}\bigl([h_1; \ldots; h_G]W_{\mathrm{fusion}} + b_{\mathrm{fusion}}\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}}.
$$

The Adaptive Gate Network transforms the regime vector into normalized group weights,
$$
w = \mathrm{Softmax}(RW_{\mathrm{gate}} + b_{\mathrm{gate}}) \in \mathbb{R}^{G}, \qquad \sum_{g=1}^{G} w_g = 1,
$$
and the resulting weights modulate the Adaptive Financial Context. Two Transformer encoder layers then process the sequence, after which the CLS representation is normalized, passed through an MLP, and linearly projected to the return forecast
$$
\hat y \in \mathbb{R}^{B}.
$$

This design is explicitly regime-adaptive: latent market-state information is not appended as an auxiliary feature, but used to modulate the relative importance of semantic feature groups inside the attention computation itself. A plausible implication is that SAFT treats regime inference and feature interaction as coupled operations rather than separate preprocessing and prediction stages.

## 2. Semantic grouping and regime-gated attention

A central component of SAFT is the partition of the 95 engineered features into $G=11$ semantic categories [2606.29347].

| Semantic category | Feature count |
|---|---:|
| Price | 7 |
| Returns | 5 |
| Volatility | 8 |
| Trend | 9 |
| Momentum | 3 |
| Volume | 8 |
| Candlestick | 11 |
| Statistics | 9 |
| Lags | 15 |
| Breakout | 15 |
| Calendar | 5 |

For each group $g$, SAFT projects the grouped sequence to head dimension $d_{\mathrm{head}}$,
$$
\Phi_g = X_g W_{\phi,g} + b_{\phi,g} \in \mathbb{R}^{B \times (L+1) \times d_{\mathrm{head}}},
$$
normalizes it,
$$
\tilde \Phi_g = \Phi_g / (\|\Phi_g\|_2 + \epsilon),
$$
and forms a cosine-similarity matrix
$$
M_g = \tilde \Phi_g \cdot \tilde \Phi_g^T \in \mathbb{R}^{B \times (L+1) \times (L+1)}.
$$
The financial bias is then aggregated as
$$
B_{\mathrm{financial}} = \sigma(\gamma)\sum_{g=1}^{G} w_g M_g \in \mathbb{R}^{B \times (L+1) \times (L+1)}.
$$

The self-attention mechanism is correspondingly modified to
$$
\mathrm{Attention}(Q,K,V)
=
\mathrm{Softmax}\!\left(
\frac{QK^T}{\sqrt{d_k}} + B_{\mathrm{temporal}} + B_{\mathrm{financial}}
\right)V.
$$
Elementwise, the attention coefficient is
$$
\alpha_{ij}
=
\frac{
\exp\!\left(
q_i^T k_j/\sqrt{d_k} + b^{\mathrm{temp}}_{ij} + \sum_{g=1}^{G} w_g m_{g,ij}
\right)
}{
\sum_{j'} \exp(\cdots)
}.
$$

The paper characterizes this mechanism as forcing the self-attention to focus on time-steps that exhibit similar behavior within each financial concept, dynamically scaled by the inferred market regime. In that sense, the term “semantic-aware” refers not to natural-language supervision but to semantically organized financial indicators whose pairwise similarities are injected directly into the attention logits.

## 3. Objective design and evaluation corrections

SAFT uses a financially-aware composite objective,
$$
\mathcal{L} = \alpha\,\mathcal{L}_{\mathrm{MSE}} + \beta\,\mathcal{L}_{\mathrm{Corr}} + \gamma\,\mathcal{L}_{\mathrm{Sign}},
$$
with
$$
\mathcal{L}_{\mathrm{MSE}} = \frac{1}{N}\sum_{i=1}^{N}(y_i-\hat y_i)^2,
$$
$$
\mathcal{L}_{\mathrm{Corr}} = 1 -
\frac{\sum_i (y_i-\bar y)(\hat y_i-\bar{\hat y})}
{\sqrt{\sum (y_i-\bar y)^2 \;\sum (\hat y_i-\bar{\hat y})^2}},
$$
and
$$
\mathcal{L}_{\mathrm{Sign}} = \frac{1}{N}\sum_{i=1}^{N}\max(0,\;-\mathrm{sgn}(y_i)\,\hat y_i).
$$
The hyperparameters $\alpha,\beta,\gamma$ are tuned via Optuna [2606.29347].

The objective is paired with two methodological corrections. First, SAFT identifies a sequence-alignment problem in prior work: using $X_{t-L:t-1}$ to predict
$$
y_t = (P_{t+h}-P_t)/P_t
$$
introduces a 1-day lookahead lag. SAFT instead aligns
$$
X_t = [x_{t-L+1},\ldots,x_t]^T \leftrightarrow y_t
$$
with no lag. Second, to avoid Sharpe inflation by $\sqrt{h}$, SAFT computes trading returns only on non-overlapping 5-day blocks,
$$
r_k = \mathrm{sign}(\hat y_{t_k})\cdot y_{t_k}, \qquad t_{k+1}=t_k+h,
$$
and then annualizes Sharpe as
$$
SR = \mu(\{r_k\})/\sigma(\{r_k\}) \cdot \sqrt{252/h}.
$$

These corrections are not peripheral implementation choices. They address reported trading performance directly, and the paper presents them as necessary safeguards against sequence misalignment and backtest inflation. A plausible implication is that SAFT should be read not only as an architectural proposal but also as a methodological intervention in how financial forecasting models are evaluated.

## 4. Experimental protocol

The reported experiments use AAPL daily data from 2018–2024, totaling 1,762 days, with chronological 70/15/15% train/validation/test splits [2606.29347]. Results are reported across five random seeds, $[42,101,2023,777,999]$, as mean $\pm 95\%$ confidence interval. Hyperparameter search uses the Optuna TPE sampler with 50 trials and MedianPruner after 5 epochs. The search space includes
- $lr \in [10^{-5},10^{-2}]$,
- $wd \in [10^{-6},10^{-1}]$,
- dropout $\in [0,0.3]$,
- $L \in \{20,40,60,80\}$,
- $d_{\mathrm{model}} \in \{32,64,96,128,160\}$,
- $d_{\mathrm{ff}} \in \{2\times,4\times\}$,
- heads $\in \{2,4,8\}$,
- layers $\in \{1,2,3\}$,
- and a Top-40 feature toggle.

The study includes ablation experiments removing the Gate Network, the Regime Encoder, or the Financial Context, as well as a comparison with all 95 features versus Top-40 features. It also reports multi-stock validation on AAPL, MSFT, GOOG, AMZN, META, and NVDA using identical hyperparameters and a single-seed run for the leaderboard. Statistical tests consist of paired $t$-tests and Cohen’s $d$ comparing Baseline AFT and Optimized AFT.

The paper’s abstract further states that experiments compare the proposed architecture against classical machine learning models, recurrent neural networks, and Transformer baselines, and that the evaluation includes chronological evaluation, ablation studies, explainability analysis, and multi-stock validation. The emphasis on chronological splitting, repeated seeds, and corrected backtesting protocols situates SAFT within a more conservative evaluation regime than is sometimes seen in financial forecasting.

## 5. Empirical findings, ablations, and explainability

On the five-seed AAPL benchmark, Baseline AFT reports
- $\mathrm{MAE}=0.0328\pm0.0031$,
- $\mathrm{RMSE}=0.0443\pm0.0028$,
- $R^2=-0.107\pm0.140$,
- $\mathrm{DA}=51.23\%\pm6.58\%$,
- $\mathrm{SR}=0.142\pm0.74$,
- and $\mathrm{Return}=+6.20\%\pm25.6\%$.

Optimized AFT with Top-40 features reports
- $\mathrm{MAE}=0.0336\pm0.0023$,
- $\mathrm{RMSE}=0.0455\pm0.0023$,
- $R^2=-0.137\pm0.117$,
- $\mathrm{DA}=51.67\%\pm6.21\%$,
- $\mathrm{SR}=0.161\pm0.64$,
- and $\mathrm{Return}=-2.89\%\pm21.9\%$.
The paired $t$-test finds no statistically significant differences, with examples including $\mathrm{DA}\ p=0.8965$ and $\mathrm{SR}\ p=0.9544$ [2606.29347].

The reported efficiency gains are more clear-cut. Parameters are reduced from $373{,}143 \rightarrow 316{,}319$ ($-15.2\%$), feature count is reduced from $95 \rightarrow 40$ ($-58\%$), and per-block complexity is given as
$$
O(LD + L^2 d_{\mathrm{model}} + G L^2),
$$
with $L=60$ and $G=11$.
The abstract correspondingly describes the model as reducing model complexity by $15.2\%$ and improving parameter efficiency through feature selection.

The multi-stock leaderboard reports the following single-seed results: AAPL achieves $\mathrm{MAE}=0.0236$, $\mathrm{DA}=62.81\%$, $\mathrm{SR}=2.286$, $\mathrm{Return}=+44.26\%$; MSFT, $\mathrm{MAE}=0.0245$, $\mathrm{DA}=56.28\%$, $\mathrm{SR}=0.478$, $\mathrm{Return}=+7.75\%$; GOOG, $\mathrm{MAE}=0.0311$, $\mathrm{DA}=49.25\%$, $\mathrm{SR}\approx0$, $\mathrm{Return}=+0.50\%$; AMZN, $\mathrm{MAE}=0.0305$, $\mathrm{DA}=54.27\%$, $\mathrm{SR}=0.712$, $\mathrm{Return}=+26.27\%$; META, $\mathrm{MAE}=0.0353$, $\mathrm{DA}=55.78\%$, $\mathrm{SR}=0.965$, $\mathrm{Return}=+18.89\%$; and NVDA, $\mathrm{MAE}=0.0666$, $\mathrm{DA}=42.21\%$, $\mathrm{SR}=-1.438$, $\mathrm{Return}=-50.69\%$. Aggregate performance across the six stocks is $\mathrm{MAE}=0.0353\pm0.0159$, $\mathrm{DA}=53.43\%\pm6.88$, $\mathrm{SR}=0.50\pm1.23$, and $\mathrm{Return}=7.83\%\pm32.75$.

The ablation results attribute a particularly strong role to the Adaptive Financial Context: removing it causes Sharpe to collapse from $0.161 \rightarrow -0.393$, while directional accuracy falls by more than 5 percentage points. Removing the Gate Network or the Regime Encoder also degrades directional accuracy and Sharpe. Explainability analysis using Integrated Gradients identifies volatility features, specifically Rolling Variance and ATR, together with Momentum-20, as having the highest attributions. The gate dynamics indicate that low-volatility regimes emphasize Price and Trend groups, whereas high-volatility regimes emphasize Volatility and Breakout groups. CLS-token attention shows a strong recency bias, with the highest weights on the last 5–10 days when forecasting.

Taken together, the quantitative picture is mixed in a way that is methodologically informative. The abstract describes competitive predictive performance, but the five-seed comparison between Baseline AFT and Optimized AFT does not show statistically significant differences on the cited metrics. By contrast, the ablations suggest that the semantic and regime-gated components are materially important to trading-oriented behavior once the architecture is fixed.

## 6. Nomenclature and relation to other 2026 uses of “SAFT”

The acronym SAFT is overloaded in the 2026 arXiv literature. In "Semantic-Topological Graph Reasoning for Language-Guided Pulmonary Screening," SAFT denotes **Selective Asymmetric Fine-Tuning**, a parameter-efficient fine-tuning strategy that updates less than 1% of parameters while freezing GroundingDINO, MedSAM, DINOv2, and most of LLaMA-3-V [2604.05620]. In "Semantic-aware Adversarial Fine-tuning for CLIP," SAFT denotes **Semantic-aware Adversarial Fine-Tuning**, a robustness-oriented procedure that fine-tunes CLIP’s image encoder using semantic-aware adversarial examples generated from ensembles of refined textual descriptions [2602.12461].

Semantic-aware AFT is distinct from both usages. It is not a PEFT recipe for multimodal medical segmentation, nor an adversarial training method for zero-shot vision-language models. Instead, it is a financial time-series Transformer architecture whose semantic awareness arises from grouping engineered indicators into Price, Returns, Volatility, Trend, Momentum, Volume, Candlestick, Statistics, Lags, Breakout, and Calendar categories, and whose adaptivity arises from regime-gated attention over those groups. This distinction is practically important, because the same acronym refers to different methodological objects: a training strategy in pulmonary screening, a fine-tuning algorithm for CLIP robustness, and a regime-adaptive forecasting architecture in finance [2606.29347].

Source: https://www.emergentmind.com/topics/semantic-aware-aft-saft