FinCAD: Adaptive Decoding for Bias-Free Backtesting

• FinCAD is an inference-time adaptation method that adjusts the decoding process to suppress pre-trained memory of future outcomes in financial backtests.
• It employs a dual-branch mechanism combining context-aware and adversarial prompts to isolate and subtract recalled signals from in-sample data.
• The approach uses entity- and date-adaptive penalty controls to selectively mitigate bias while preserving the model's context-grounded reasoning.

FinCAD is an inference-time adaptation of Context-Aware Decoding for mitigating parametric look-ahead bias in financial backtesting with LLMs. Parametric look-ahead bias arises when an LLM used on historical financial data has already “seen” in pretraining the price movements or corporate events that the backtest is intended to evaluate, so the model’s weights contain in-sample information that leaks into trading signals. The method is designed to suppress an LLM’s memory of historical outcomes without retraining by pairing an adversarial bias-discovery pipeline with an entity- and date-adaptive decoding rule that penalizes memorized in-sample dates and decays to zero out-of-sample (Li et al., 23 May 2026).

1. Problem formulation and motivation

In the formulation associated with FinCAD, an LLM trained through 2024 may already “know” which way 2018–2020 stocks moved, so backtests over windows overlapping the model’s training cutoff can falsely inflate returns. Traditional controls for survivorship, data-snooping, and pipeline leakage audit the data feed, but they do not detect bias that resides inside the model parameters. This creates a setting in which long-horizon backtests become systematically misleading, while the only intrinsically bias-free period is the short interval after the pretraining cutoff (Li et al., 23 May 2026).

The stated objective is an inference-time correction rather than retraining or weight editing. FinCAD is therefore positioned as a decoding-time intervention that attempts to make the model “forget” memorized future outcomes while retaining context-grounded reasoning. This design choice is significant because it targets evaluation integrity under fixed model weights, which is especially relevant when the LLM is already deployed or cannot be modified at the parameter level.

A plausible implication is that FinCAD addresses a failure mode specific to foundation-model backtesting rather than a generic defect in quantitative finance pipelines. The method treats memorization as a conditional decoding artifact that can be probed and penalized, rather than as a property requiring irreversible model surgery.

2. Core decoding mechanism

FinCAD repurposes Context-Aware Decoding by constructing two prompts for a stock symbol $s$ on date $t$. The context branch is defined as

$$x_{\rm ctx} = T_{\rm ctx}\;\Vert\; (s,t)\;\Vert\;F_{\rm task}\;\Vert\;D(s,t),$$

where $T_{\rm ctx}$ is the system prompt, $F_{\rm task}$ is the JSON-output instruction, and $D(s,t)$ is the price-derived summary at time $t$. The prior branch is

$$x_{\rm prior} = T_{\rm prior}^* \;\Vert\; (s,t)\;\Vert\;F_{\rm task},$$

where $T_{\rm prior}^*$ is an adversarially optimized prompt that elicits parametric recall of the future without supplying market data (Li et al., 23 May 2026).

The adjusted next-token distribution is

$$\tilde y_{s,t} = \mathrm{softmax}\Bigl(\,(1+\alpha(s,t))\,\mathcal M(x_{\rm ctx}) \;-\;\alpha(s,t)\,\mathcal M(x_{\rm prior})\Bigr),$$

with $t$0 denoting model logits and $t$1 controlling subtraction strength. In this construction, the context-conditioned branch carries the task-relevant evidence, whereas the prior branch is intended to expose memorized directional knowledge. The subtraction is therefore a logit-level suppression of recalled outcomes rather than a post hoc adjustment of decisions.

This formulation is technically important because it isolates two sources of signal: information grounded in the supplied date-specific context, and information recoverable from parametric memory alone. FinCAD uses the latter as an “oracle” to be subtracted. The paper explicitly contrasts this fine-grained, logit-level subtraction with simpler mitigations such as anonymization and prompt-injection, which did not improve ranking fidelity in the reported experiments.

3. Adversarial bias-discovery pipeline

A central component of FinCAD is the optimization of the prior instruction $t$2. The calibration set is

$t$3

drawn from 2005–2015 S&P 500 history. The calibration task $t$4 asks: “Did stock $t$5 go up or down after date $t$6? Answer from memory only.” For a candidate instruction $t$7, the model prediction is

$t$8

and the objective is to maximize directional accuracy over the calibration set: $t$9

In practice, MIPROv2 is used to propose and rank candidate instructions, and the highest-accuracy prompt on a held-out validation split is selected (Li et al., 23 May 2026). The resulting prompt is described as a short natural-language instruction, model-specific, that reliably activates memorized directional recall.

This pipeline matters because the effectiveness of the subtraction step depends on whether the prior branch can consistently expose the model’s latent knowledge of future movements. If the prior prompt under-activates recall, the subtraction would be weak or misaligned. The paper’s framing therefore treats bias discovery as an optimization problem over prompts rather than as a manually engineered template.

4. Entity- and date-adaptive control of the penalty

FinCAD does not apply a uniform penalty across assets or dates. Instead, $$x_{\rm ctx} = T_{\rm ctx}\;\Vert\; (s,t)\;\Vert\;F_{\rm task}\;\Vert\;D(s,t),$$0 is set through a two-step probe designed to estimate per-entity and per-date memorization. The completion-based probe uses

$$x_{\rm ctx} = T_{\rm ctx}\;\Vert\; (s,t)\;\Vert\;F_{\rm task}\;\Vert\;D(s,t),$$1

from which up/down logits $$x_{\rm ctx} = T_{\rm ctx}\;\Vert\; (s,t)\;\Vert\;F_{\rm task}\;\Vert\;D(s,t),$$2 are read. These define

$$x_{\rm ctx} = T_{\rm ctx}\;\Vert\; (s,t)\;\Vert\;F_{\rm task}\;\Vert\;D(s,t),$$3

The normalized binary entropy $$x_{\rm ctx} = T_{\rm ctx}\;\Vert\; (s,t)\;\Vert\;F_{\rm task}\;\Vert\;D(s,t),$$4 serves as a confidence-sensitive probe of directional recall (Li et al., 23 May 2026).

Before backtesting each entity $$x_{\rm ctx} = T_{\rm ctx}\;\Vert\; (s,t)\;\Vert\;F_{\rm task}\;\Vert\;D(s,t),$$5, the method probes fixed calibration dates $$x_{\rm ctx} = T_{\rm ctx}\;\Vert\; (s,t)\;\Vert\;F_{\rm task}\;\Vert\;D(s,t),$$6 to compute

$$x_{\rm ctx} = T_{\rm ctx}\;\Vert\; (s,t)\;\Vert\;F_{\rm task}\;\Vert\;D(s,t),$$7

Here, $$x_{\rm ctx} = T_{\rm ctx}\;\Vert\; (s,t)\;\Vert\;F_{\rm task}\;\Vert\;D(s,t),$$8 measures date-sensitivity of the model’s confidence: high $$x_{\rm ctx} = T_{\rm ctx}\;\Vert\; (s,t)\;\Vert\;F_{\rm task}\;\Vert\;D(s,t),$$9 indicates genuine temporal memorization, whereas low $T_{\rm ctx}$0 indicates a stable brand prior. Two scalars are then defined: $T_{\rm ctx}$1 with $T_{\rm ctx}$2 and $T_{\rm ctx}$3 estimated once through model-specific profiling. The final penalty is

$T_{\rm ctx}$4

The reported properties of this rule are threefold: it is entity-adaptive, date-selective, and out-of-sample transparent. In particular, for out-of-sample dates, $T_{\rm ctx}$5 so $T_{\rm ctx}$6, implying $T_{\rm ctx}$7. This suggests that the method is intended to activate only when unusually low entropy indicates likely memorization, rather than imposing indiscriminate suppression across the entire evaluation horizon.

5. Inference-time workflow

The operational procedure begins by building the context and prior prompts, then running the adaptive probe to compute $T_{\rm ctx}$8, $T_{\rm ctx}$9, $F_{\rm task}$0, and $F_{\rm task}$1. Decoding then proceeds autoregressively: at each step, logits from the context branch and prior branch are combined as

$F_{\rm task}$2

a token is sampled from the softmax of the blended logits, and the generated token is appended to both branches and to the output sequence (Li et al., 23 May 2026).

A further implementation detail is the handling of formatting failures. If the output fails JSON parsing, the procedure reduces $F_{\rm task}$3 and retries, up to five times. The return value is the parsed JSON decision. This shows that the decoding intervention is integrated into a structured-output trading workflow rather than operating as an isolated classification primitive.

The computational implications are explicit. FinCAD introduces one extra forward pass per token plus one probe per decision date. The overhead is described as moderate but not negligible for large portfolios. That characterization places the method between trivial prompt-only mitigations and more expensive retraining-based alternatives.

6. Empirical evaluation, limitations, and significance

The evaluation comprises three experiments: Reasoning Preservation, Honesty Drop, and Ranking Alignment. The following summary organizes the reported setup and findings (Li et al., 23 May 2026).

Experiment	Setup	Reported finding
Reasoning Preservation	Five instruction-tuned LLMs; FinEval, MMLU-Pro, GSM8K, Competition-Math, HumanEval	Mean accuracy changed by at most $F_{\rm task}$4 points / $F_{\rm task}$5 points; four of five within $F_{\rm task}$6 points
Honesty Drop	Single-stock backtests on NVDA, MSFT, AAPL, NFLX, AMZN; in-sample 2010–2020; out-of-sample 2025–2026	Qwen2.5-14B: mean in-sample return drop $F_{\rm task}$7, mean Sharpe drop $F_{\rm task}$8, peaking at $F_{\rm task}$9 on NVDA; Phi-4-14B: mean return $D(s,t)$0, Sharpe $D(s,t)$1
Ranking Alignment	Eleven LLMs on SPY backtest	Spearman correlation rose from $D(s,t)$2 to $D(s,t)$3, with one-sided $D(s,t)$4

The Reasoning Preservation results are used to argue that the backtest effects are not explained by destructive suppression of general capabilities. The Honesty Drop experiment shows substantial in-sample deterioration on heavily memorized equities, which the paper interprets as removal of look-ahead contamination rather than a loss of valid predictive skill. For out-of-sample evaluation, Phi-4 is reported to maintain byte-identical trajectories with $D(s,t)$5, while Qwen2.5-14B has mild $D(s,t)$6 and changes of at most $D(s,t)$7 and Sharpe within $D(s,t)$8 of baseline across the evaluated setting (Li et al., 23 May 2026).

The Ranking Alignment experiment is central to the paper’s broader claim. Baseline in-sample rankings already correlate with out-of-sample rankings at $D(s,t)$9, but FinCAD raises this to $t$0. The paper further reports that anonymization and prompt-injection both lowered $t$1, which is presented as evidence that the fine-grained subtraction mechanism is necessary for reliable alignment between in-sample and future performance.

The limitations are explicit. First, the method is only a partial mitigation: logit subtraction cannot guarantee full erasure of look-ahead knowledge, and residual memorization may persist if the prior prompt or probe fails to activate it. The reported in-sample drop is therefore described as a lower bound on true contamination. Second, the evaluation is confined to 7–14B parameter LLMs, leaving extension to frontier-scale or closed-source models open. Third, the experiments focus on U.S. equities, including mega-, mid-, and small-cap settings as well as SPY, so applying the method to fixed income, FX, crypto, or other domains would require fresh calibration of $t$2. A plausible implication is that FinCAD should be understood less as a universal decontamination guarantee than as a calibrated inference-time control mechanism for detecting and suppressing memorized historical outcomes under specific deployment conditions.