Semantic-aware AFT for Stock Prediction

• Semantic-aware AFT is a regime-adaptive Transformer model that groups 95 financial features into 11 semantic categories to forecast stock returns.
• It integrates a Market Regime Encoder, Adaptive Gate Network, and Adaptive Financial Context module to dynamically bias self-attention using semantic similarity.
• The architecture employs a composite objective and corrected backtesting protocols, with ablation studies showing that removing semantic components degrades key performance metrics like directional accuracy and Sharpe ratio.

Semantic-aware AFT (SAFT) is a Transformer-based architecture for stock return prediction under non-stationary financial markets. In the formulation described in "Adaptive Financial Transformer with Regime-Gated Attention for Stock Return Prediction," SAFT extends a standard Transformer encoder with a Market Regime Encoder, an Adaptive Gate Network, and an Adaptive Financial Context module, allowing self-attention to be dynamically biased by semantic relationships between financial indicators rather than treating all input features uniformly. The model groups 95 engineered financial features into 11 semantic categories, uses corrected sequence alignment and non-overlapping backtesting, and optimizes a financially-aware composite objective that combines prediction error, directional accuracy, and non-overlapping Sharpe ratio (Sarkar, 28 Jun 2026).

1. Architectural formulation

SAFT is defined as an extension of the standard Transformer encoder with three domain-specific modules: a Market Regime Encoder that compresses group-wise feature statistics into a latent regime vector, an Adaptive Gate Network that turns the regime vector into per-group gate weights, and an Adaptive Financial Context that uses those weights to bias the self-attention scores via group-wise semantic similarity (Sarkar, 28 Jun 2026).

The input is a batch of lookback windows

$X \in \mathbb{R}^{B \times L \times D}$

with $D=95$ technical features over $L=60$ trading days. The sequence is projected to

$$E = XW_{\mathrm{embed}} + b_{\mathrm{embed}} \in \mathbb{R}^{B \times L \times d_{\mathrm{model}}},$$

prepended with a learnable CLS token, and added to sinusoidal positional encoding. The Market Regime Encoder operates on each semantic feature group $X_g$ and computes

$$h_g = \mathrm{Pool}\bigl(\mathrm{GELU}(X_g W_g + b_g)\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}},$$

followed by concatenation and fusion:

$$R = \mathrm{GELU}\bigl([h_1; \ldots; h_G]W_{\mathrm{fusion}} + b_{\mathrm{fusion}}\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}}.$$

The Adaptive Gate Network transforms the regime vector into normalized group weights,

$$w = \mathrm{Softmax}(RW_{\mathrm{gate}} + b_{\mathrm{gate}}) \in \mathbb{R}^{G}, \qquad \sum_{g=1}^{G} w_g = 1,$$

and the resulting weights modulate the Adaptive Financial Context. Two Transformer encoder layers then process the sequence, after which the CLS representation is normalized, passed through an MLP, and linearly projected to the return forecast

$\hat y \in \mathbb{R}^{B}.$

This design is explicitly regime-adaptive: latent market-state information is not appended as an auxiliary feature, but used to modulate the relative importance of semantic feature groups inside the attention computation itself. A plausible implication is that SAFT treats regime inference and feature interaction as coupled operations rather than separate preprocessing and prediction stages.

2. Semantic grouping and regime-gated attention

A central component of SAFT is the partition of the 95 engineered features into $G=11$ semantic categories (Sarkar, 28 Jun 2026).

Semantic category	Feature count
Price	7
Returns	5
Volatility	8
Trend	9
Momentum	3
Volume	8
Candlestick	11
Statistics	9
Lags	15
Breakout	15
Calendar	5

For each group $D=95$0, SAFT projects the grouped sequence to head dimension $D=95$1,

$D=95$2

normalizes it,

$D=95$3

and forms a cosine-similarity matrix

$D=95$4

The financial bias is then aggregated as

$D=95$5

The self-attention mechanism is correspondingly modified to

$D=95$6

Elementwise, the attention coefficient is

$D=95$7

The paper characterizes this mechanism as forcing the self-attention to focus on time-steps that exhibit similar behavior within each financial concept, dynamically scaled by the inferred market regime. In that sense, the term “semantic-aware” refers not to natural-language supervision but to semantically organized financial indicators whose pairwise similarities are injected directly into the attention logits.

3. Objective design and evaluation corrections

SAFT uses a financially-aware composite objective,

$D=95$8

with

$D=95$9

$L=60$0

and

$L=60$1

The hyperparameters $L=60$2 are tuned via Optuna (Sarkar, 28 Jun 2026).

The objective is paired with two methodological corrections. First, SAFT identifies a sequence-alignment problem in prior work: using $L=60$3 to predict

$L=60$4

introduces a 1-day lookahead lag. SAFT instead aligns

$L=60$5

with no lag. Second, to avoid Sharpe inflation by $L=60$6, SAFT computes trading returns only on non-overlapping 5-day blocks,

$L=60$7

and then annualizes Sharpe as

$L=60$8

These corrections are not peripheral implementation choices. They address reported trading performance directly, and the paper presents them as necessary safeguards against sequence misalignment and backtest inflation. A plausible implication is that SAFT should be read not only as an architectural proposal but also as a methodological intervention in how financial forecasting models are evaluated.

4. Experimental protocol

The reported experiments use AAPL daily data from 2018–2024, totaling 1,762 days, with chronological 70/15/15% train/validation/test splits (Sarkar, 28 Jun 2026). Results are reported across five random seeds, $L=60$9, as mean $$E = XW_{\mathrm{embed}} + b_{\mathrm{embed}} \in \mathbb{R}^{B \times L \times d_{\mathrm{model}}},$$0 confidence interval. Hyperparameter search uses the Optuna TPE sampler with 50 trials and MedianPruner after 5 epochs. The search space includes

• $$E = XW_{\mathrm{embed}} + b_{\mathrm{embed}} \in \mathbb{R}^{B \times L \times d_{\mathrm{model}}},$$1,
• $$E = XW_{\mathrm{embed}} + b_{\mathrm{embed}} \in \mathbb{R}^{B \times L \times d_{\mathrm{model}}},$$2,
• dropout $$E = XW_{\mathrm{embed}} + b_{\mathrm{embed}} \in \mathbb{R}^{B \times L \times d_{\mathrm{model}}},$$3,
• $$E = XW_{\mathrm{embed}} + b_{\mathrm{embed}} \in \mathbb{R}^{B \times L \times d_{\mathrm{model}}},$$4,
• $$E = XW_{\mathrm{embed}} + b_{\mathrm{embed}} \in \mathbb{R}^{B \times L \times d_{\mathrm{model}}},$$5,
• $$E = XW_{\mathrm{embed}} + b_{\mathrm{embed}} \in \mathbb{R}^{B \times L \times d_{\mathrm{model}}},$$6,
• heads $$E = XW_{\mathrm{embed}} + b_{\mathrm{embed}} \in \mathbb{R}^{B \times L \times d_{\mathrm{model}}},$$7,
• layers $$E = XW_{\mathrm{embed}} + b_{\mathrm{embed}} \in \mathbb{R}^{B \times L \times d_{\mathrm{model}}},$$8,
• and a Top-40 feature toggle.

The study includes ablation experiments removing the Gate Network, the Regime Encoder, or the Financial Context, as well as a comparison with all 95 features versus Top-40 features. It also reports multi-stock validation on AAPL, MSFT, GOOG, AMZN, META, and NVDA using identical hyperparameters and a single-seed run for the leaderboard. Statistical tests consist of paired $$E = XW_{\mathrm{embed}} + b_{\mathrm{embed}} \in \mathbb{R}^{B \times L \times d_{\mathrm{model}}},$$9-tests and Cohen’s $X_g$0 comparing Baseline AFT and Optimized AFT.

The paper’s abstract further states that experiments compare the proposed architecture against classical machine learning models, recurrent neural networks, and Transformer baselines, and that the evaluation includes chronological evaluation, ablation studies, explainability analysis, and multi-stock validation. The emphasis on chronological splitting, repeated seeds, and corrected backtesting protocols situates SAFT within a more conservative evaluation regime than is sometimes seen in financial forecasting.

5. Empirical findings, ablations, and explainability

On the five-seed AAPL benchmark, Baseline AFT reports

• $X_g$1,
• $X_g$2,
• $X_g$3,
• $X_g$4,
• $X_g$5,
• and $X_g$6.

Optimized AFT with Top-40 features reports

• $X_g$7,
• $X_g$8,
• $X_g$9,
• $$h_g = \mathrm{Pool}\bigl(\mathrm{GELU}(X_g W_g + b_g)\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}},$$0,
• $$h_g = \mathrm{Pool}\bigl(\mathrm{GELU}(X_g W_g + b_g)\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}},$$1,
• and $$h_g = \mathrm{Pool}\bigl(\mathrm{GELU}(X_g W_g + b_g)\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}},$$2. The paired $$h_g = \mathrm{Pool}\bigl(\mathrm{GELU}(X_g W_g + b_g)\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}},$$3-test finds no statistically significant differences, with examples including $$h_g = \mathrm{Pool}\bigl(\mathrm{GELU}(X_g W_g + b_g)\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}},$$4 and $$h_g = \mathrm{Pool}\bigl(\mathrm{GELU}(X_g W_g + b_g)\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}},$$5 (Sarkar, 28 Jun 2026).

The reported efficiency gains are more clear-cut. Parameters are reduced from $$h_g = \mathrm{Pool}\bigl(\mathrm{GELU}(X_g W_g + b_g)\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}},$$6 ($$h_g = \mathrm{Pool}\bigl(\mathrm{GELU}(X_g W_g + b_g)\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}},$$7), feature count is reduced from $$h_g = \mathrm{Pool}\bigl(\mathrm{GELU}(X_g W_g + b_g)\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}},$$8 ($$h_g = \mathrm{Pool}\bigl(\mathrm{GELU}(X_g W_g + b_g)\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}},$$9), and per-block complexity is given as

$$R = \mathrm{GELU}\bigl([h_1; \ldots; h_G]W_{\mathrm{fusion}} + b_{\mathrm{fusion}}\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}}.$$0

with $$R = \mathrm{GELU}\bigl([h_1; \ldots; h_G]W_{\mathrm{fusion}} + b_{\mathrm{fusion}}\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}}.$$1 and $$R = \mathrm{GELU}\bigl([h_1; \ldots; h_G]W_{\mathrm{fusion}} + b_{\mathrm{fusion}}\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}}.$$2. The abstract correspondingly describes the model as reducing model complexity by $$R = \mathrm{GELU}\bigl([h_1; \ldots; h_G]W_{\mathrm{fusion}} + b_{\mathrm{fusion}}\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}}.$$3 and improving parameter efficiency through feature selection.

The multi-stock leaderboard reports the following single-seed results: AAPL achieves $$R = \mathrm{GELU}\bigl([h_1; \ldots; h_G]W_{\mathrm{fusion}} + b_{\mathrm{fusion}}\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}}.$$4, $$R = \mathrm{GELU}\bigl([h_1; \ldots; h_G]W_{\mathrm{fusion}} + b_{\mathrm{fusion}}\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}}.$$5, $$R = \mathrm{GELU}\bigl([h_1; \ldots; h_G]W_{\mathrm{fusion}} + b_{\mathrm{fusion}}\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}}.$$6, $$R = \mathrm{GELU}\bigl([h_1; \ldots; h_G]W_{\mathrm{fusion}} + b_{\mathrm{fusion}}\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}}.$$7; MSFT, $$R = \mathrm{GELU}\bigl([h_1; \ldots; h_G]W_{\mathrm{fusion}} + b_{\mathrm{fusion}}\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}}.$$8, $$R = \mathrm{GELU}\bigl([h_1; \ldots; h_G]W_{\mathrm{fusion}} + b_{\mathrm{fusion}}\bigr) \in \mathbb{R}^{B \times d_{\mathrm{regime}}}.$$9, $$w = \mathrm{Softmax}(RW_{\mathrm{gate}} + b_{\mathrm{gate}}) \in \mathbb{R}^{G}, \qquad \sum_{g=1}^{G} w_g = 1,$$0, $$w = \mathrm{Softmax}(RW_{\mathrm{gate}} + b_{\mathrm{gate}}) \in \mathbb{R}^{G}, \qquad \sum_{g=1}^{G} w_g = 1,$$1; GOOG, $$w = \mathrm{Softmax}(RW_{\mathrm{gate}} + b_{\mathrm{gate}}) \in \mathbb{R}^{G}, \qquad \sum_{g=1}^{G} w_g = 1,$$2, $$w = \mathrm{Softmax}(RW_{\mathrm{gate}} + b_{\mathrm{gate}}) \in \mathbb{R}^{G}, \qquad \sum_{g=1}^{G} w_g = 1,$$3, $$w = \mathrm{Softmax}(RW_{\mathrm{gate}} + b_{\mathrm{gate}}) \in \mathbb{R}^{G}, \qquad \sum_{g=1}^{G} w_g = 1,$$4, $$w = \mathrm{Softmax}(RW_{\mathrm{gate}} + b_{\mathrm{gate}}) \in \mathbb{R}^{G}, \qquad \sum_{g=1}^{G} w_g = 1,$$5; AMZN, $$w = \mathrm{Softmax}(RW_{\mathrm{gate}} + b_{\mathrm{gate}}) \in \mathbb{R}^{G}, \qquad \sum_{g=1}^{G} w_g = 1,$$6, $$w = \mathrm{Softmax}(RW_{\mathrm{gate}} + b_{\mathrm{gate}}) \in \mathbb{R}^{G}, \qquad \sum_{g=1}^{G} w_g = 1,$$7, $$w = \mathrm{Softmax}(RW_{\mathrm{gate}} + b_{\mathrm{gate}}) \in \mathbb{R}^{G}, \qquad \sum_{g=1}^{G} w_g = 1,$$8, $$w = \mathrm{Softmax}(RW_{\mathrm{gate}} + b_{\mathrm{gate}}) \in \mathbb{R}^{G}, \qquad \sum_{g=1}^{G} w_g = 1,$$9; META, $\hat y \in \mathbb{R}^{B}.$0, $\hat y \in \mathbb{R}^{B}.$1, $\hat y \in \mathbb{R}^{B}.$2, $\hat y \in \mathbb{R}^{B}.$3; and NVDA, $\hat y \in \mathbb{R}^{B}.$4, $\hat y \in \mathbb{R}^{B}.$5, $\hat y \in \mathbb{R}^{B}.$6, $\hat y \in \mathbb{R}^{B}.$7. Aggregate performance across the six stocks is $\hat y \in \mathbb{R}^{B}.$8, $\hat y \in \mathbb{R}^{B}.$9, $G=11$0, and $G=11$1.

The ablation results attribute a particularly strong role to the Adaptive Financial Context: removing it causes Sharpe to collapse from $G=11$2, while directional accuracy falls by more than 5 percentage points. Removing the Gate Network or the Regime Encoder also degrades directional accuracy and Sharpe. Explainability analysis using Integrated Gradients identifies volatility features, specifically Rolling Variance and ATR, together with Momentum-20, as having the highest attributions. The gate dynamics indicate that low-volatility regimes emphasize Price and Trend groups, whereas high-volatility regimes emphasize Volatility and Breakout groups. CLS-token attention shows a strong recency bias, with the highest weights on the last 5–10 days when forecasting.

Taken together, the quantitative picture is mixed in a way that is methodologically informative. The abstract describes competitive predictive performance, but the five-seed comparison between Baseline AFT and Optimized AFT does not show statistically significant differences on the cited metrics. By contrast, the ablations suggest that the semantic and regime-gated components are materially important to trading-oriented behavior once the architecture is fixed.

6. Nomenclature and relation to other 2026 uses of “SAFT”

The acronym SAFT is overloaded in the 2026 arXiv literature. In "Semantic-Topological Graph Reasoning for Language-Guided Pulmonary Screening," SAFT denotes Selective Asymmetric Fine-Tuning, a parameter-efficient fine-tuning strategy that updates less than 1% of parameters while freezing GroundingDINO, MedSAM, DINOv2, and most of LLaMA-3-V (Xue et al., 7 Apr 2026). In "Semantic-aware Adversarial Fine-tuning for CLIP," SAFT denotes Semantic-aware Adversarial Fine-Tuning, a robustness-oriented procedure that fine-tunes CLIP’s image encoder using semantic-aware adversarial examples generated from ensembles of refined textual descriptions (Zhang et al., 12 Feb 2026).

Semantic-aware AFT is distinct from both usages. It is not a PEFT recipe for multimodal medical segmentation, nor an adversarial training method for zero-shot vision-LLMs. Instead, it is a financial time-series Transformer architecture whose semantic awareness arises from grouping engineered indicators into Price, Returns, Volatility, Trend, Momentum, Volume, Candlestick, Statistics, Lags, Breakout, and Calendar categories, and whose adaptivity arises from regime-gated attention over those groups. This distinction is practically important, because the same acronym refers to different methodological objects: a training strategy in pulmonary screening, a fine-tuning algorithm for CLIP robustness, and a regime-adaptive forecasting architecture in finance (Sarkar, 28 Jun 2026).