Feel-Good Thompson Sampling (FG-TS)

• FG-TS is a family of posterior-sampling algorithms that tilts the posterior toward models predicting high rewards, enhancing exploration in uncertain environments.
• It introduces an optimism term into the loss function to correct under-exploration in standard Thompson Sampling, yielding minimax-regret bounds in contextual bandits and reinforcement learning.
• Variants such as Smoothed FG-TS and BART-based adaptations modify the mechanism for improved stability and nonparametric applicability, though they can be sensitive to tuning and sampling accuracy.

Searching arXiv for recent and foundational papers on Feel-Good Thompson Sampling. Feel-Good Thompson Sampling (FG-TS) is a family of posterior-sampling algorithms in which the posterior or pseudo-posterior is deliberately tilted toward models that predict high rewards, thereby injecting optimism into Thompson Sampling without adding a separate UCB-style action bonus. In the foundational formulation for contextual bandits and reinforcement learning, the modification is introduced to address a frequentist weakness of standard Thompson Sampling: in pessimistic or weakly identifying environments, the unmodified posterior can under-explore and incur suboptimal regret (Zhang, 2021). Subsequent work has extended the same idea to approximate posterior sampling, smoothed objectives, contextual dueling bandits, nonparametric BART-based bandits, variance-aware contextual bandits, and reinforcement learning with approximate sampling, while also clarifying where the method is effective and where it is sensitive to posterior approximation and tuning (Anand et al., 21 Jul 2025, Li et al., 2024, Deng et al., 8 Feb 2026, Li et al., 3 Nov 2025, Ishfaq et al., 2024).

1. Origins and motivation

FG-TS arose from a frequentist analysis of contextual bandits in which the standard Thompson Sampling update is based on a prior $p_0(\theta)$, a model value function $f(\theta,x,a)$, and a loss $L(\theta,x,a,r)$, producing the posterior

$$p(\theta \mid S_{t-1}) \propto \exp\Big(-\sum_{s=1}^{t-1} L(\theta,x_s,a_s,r_s)\Big) p_0(\theta).$$

Under the canonical squared-loss choice

$L(\theta,x,a,r)=\eta\big(f(\theta,x,a)-r\big)^2,$

standard Thompson Sampling samples $\theta_t$ from this posterior and plays the greedy action $a_t\in\arg\max_a f(\theta_t,x_t,a)$ (Zhang, 2021).

The foundational analysis isolates a failure mode in which standard Thompson Sampling is “not aggressive enough” in exploration. In a two-action, finite-model example with a uniform prior over $N$ models, the paper proves

$$\sum_{t=1}^T \mathbb{E}[\Delta_t] \ge 0.5\,T(1-1/N)^T,$$

so for $T=N$ the regret is $f(\theta,x,a)$0, even though the minimax rate in the finite-class setting is $f(\theta,x,a)$1 (Zhang, 2021). The later empirical study frames the same issue more broadly: recent theory shows that standard Thompson Sampling “does not explore aggressively enough in high-dimensional problems,” and FG-TS is designed to correct that behavior by biasing the posterior toward high-reward models (Anand et al., 21 Jul 2025).

This motivation persists across later variants. In the BART-based nonparametric setting, the authors state that “standard Thompson sampling can under-explore and is difficult to analyze in nonparametric classes,” which is why a feel-good variant is introduced as a technical companion to Bayesian Forest Thompson Sampling (BFTS) (Deng et al., 8 Feb 2026). In reinforcement learning, the same idea appears as a way to encourage deep exploration by favoring $f(\theta,x,a)$2-functions that assign high value to the current initial state (Ishfaq et al., 2024).

2. Posterior tilting and the core “feel-good” mechanism

The defining modification of FG-TS is the insertion of an optimism term directly into the posterior weight. In the contextual-bandit formulation, the per-sample loss becomes

$f(\theta,x,a)$3

where $f(\theta,x,a)$4, $f(\theta,x,a)$5 is the squared-loss weight, $f(\theta,x,a)$6 controls the strength of the feel-good bias, and $f(\theta,x,a)$7 truncates extreme predictions (Zhang, 2021). The resulting posterior favors models whose greedy values on observed contexts are large.

A closely related contextual-bandit presentation writes the FG-TS loss as

$f(\theta,x,a)$8

with posterior

$f(\theta,x,a)$9

where $L(\theta,x,a,r)$0 is the cumulative FG loss. The decision rule is unchanged from standard Thompson Sampling: $L(\theta,x,a,r)$1 so there is “no extra UCB-style bonus at decision time”; the optimism is injected into the posterior itself (Anand et al., 21 Jul 2025).

Two structural variants are prominent. First, “Smoothed Feel-Good Thompson Sampling” (SFG-TS) replaces the nonsmooth $L(\theta,x,a,r)$2 term by a differentiable surrogate,

$L(\theta,x,a,r)$3

where $L(\theta,x,a,r)$4 and

$L(\theta,x,a,r)$5

This smoothing is introduced specifically to make Langevin- and HMC-type samplers more stable (Anand et al., 21 Jul 2025).

Second, the BART-based FG-TS variant used inside BFTS keeps the same arm-wise BART posterior machinery as standard BFTS but adds a feel-good loss

$L(\theta,x,a,r)$6

and in practice implements the optimism through a reweighting of posterior draws by $L(\theta,x,a,r)$7, where

$L(\theta,x,a,r)$8

Setting $L(\theta,x,a,r)$9 recovers standard BFTS, and the theoretically optimal $$p(\theta \mid S_{t-1}) \propto \exp\Big(-\sum_{s=1}^{t-1} L(\theta,x_s,a_s,r_s)\Big) p_0(\theta).$$0 is of order $$p(\theta \mid S_{t-1}) \propto \exp\Big(-\sum_{s=1}^{t-1} L(\theta,x_s,a_s,r_s)\Big) p_0(\theta).$$1, so the authors describe the feel-good modification as a “vanishing perturbation” of BFTS (Deng et al., 8 Feb 2026).

3. Foundational theory: regret decomposition and decoupling

The original theory rewrites instantaneous regret as the difference between a model-fit term and an optimism term. With $$p(\theta \mid S_{t-1}) \propto \exp\Big(-\sum_{s=1}^{t-1} L(\theta,x_s,a_s,r_s)\Big) p_0(\theta).$$2, the decomposition is

$$p(\theta \mid S_{t-1}) \propto \exp\Big(-\sum_{s=1}^{t-1} L(\theta,x_s,a_s,r_s)\Big) p_0(\theta).$$3

where $$p(\theta \mid S_{t-1}) \propto \exp\Big(-\sum_{s=1}^{t-1} L(\theta,x_s,a_s,r_s)\Big) p_0(\theta).$$4 is a Bellman-error-like prediction term and $$p(\theta \mid S_{t-1}) \propto \exp\Big(-\sum_{s=1}^{t-1} L(\theta,x_s,a_s,r_s)\Big) p_0(\theta).$$5 is the feel-good exploration term (Zhang, 2021). This identity is the basis for the later literature: the optimism term does not merely alter action probabilities, but appears explicitly with a negative sign in the regret bound.

To control the dependence between sampled model and chosen action, the theory introduces a decoupling coefficient $$p(\theta \mid S_{t-1}) \propto \exp\Big(-\sum_{s=1}^{t-1} L(\theta,x_s,a_s,r_s)\Big) p_0(\theta).$$6. In the finite-action contextual-bandit case, $$p(\theta \mid S_{t-1}) \propto \exp\Big(-\sum_{s=1}^{t-1} L(\theta,x_s,a_s,r_s)\Big) p_0(\theta).$$7 when $$p(\theta \mid S_{t-1}) \propto \exp\Big(-\sum_{s=1}^{t-1} L(\theta,x_s,a_s,r_s)\Big) p_0(\theta).$$8. In the linearly embeddable contextual-bandit model

$$p(\theta \mid S_{t-1}) \propto \exp\Big(-\sum_{s=1}^{t-1} L(\theta,x_s,a_s,r_s)\Big) p_0(\theta).$$9

the same coefficient is bounded by the embedding dimension $L(\theta,x,a,r)=\eta\big(f(\theta,x,a)-r\big)^2,$0 (Zhang, 2021). This converts bandit regret into an online least-squares estimation problem plus a complexity term.

The resulting frequentist guarantees are minimax-rate up to logarithmic factors. For finite-action contextual bandits with a finite function class of size $L(\theta,x,a,r)=\eta\big(f(\theta,x,a)-r\big)^2,$1, FG-TS attains

$L(\theta,x,a,r)=\eta\big(f(\theta,x,a)-r\big)^2,$2

in the realizable case. For linearly embeddable bandits with parametric complexity $L(\theta,x,a,r)=\eta\big(f(\theta,x,a)-r\big)^2,$3 and embedding dimension $L(\theta,x,a,r)=\eta\big(f(\theta,x,a)-r\big)^2,$4, the bound is

$L(\theta,x,a,r)=\eta\big(f(\theta,x,a)-r\big)^2,$5

and for standard linear bandits, where $L(\theta,x,a,r)=\eta\big(f(\theta,x,a)-r\big)^2,$6, this becomes

$L(\theta,x,a,r)=\eta\big(f(\theta,x,a)-r\big)^2,$7

which the paper identifies as minimax-optimal up to logarithmic factors (Zhang, 2021).

The same paper also separates this frequentist story from the Bayesian one. Standard Thompson Sampling retains a Bayesian-regret guarantee, but the feel-good modification is what restores minimax-style frequentist control in the worst case (Zhang, 2021). A common misconception is therefore that FG-TS is merely a heuristic exploration bias; in the foundational treatment, the feel-good term is the object that makes the frequentist analysis go through.

4. Major variants and domain-specific formulations

Later work preserved the posterior-tilting idea but changed the form of the feel-good term to match the geometry of the problem class.

Setting	Feel-good construction	Reported guarantee or role
Contextual bandits	Posterior tilt by $L(\theta,x,a,r)=\eta\big(f(\theta,x,a)-r\big)^2,$8 or $L(\theta,x,a,r)=\eta\big(f(\theta,x,a)-r\big)^2,$9	Minimax-rate frequentist regret up to logs in finite-action and linear cases
Contextual dueling bandits	Two independent pseudo-posteriors with a dueling-specific max-difference bonus	Nearly minimax-optimal $\theta_t$0 regret
Reinforcement learning	Feel-good prior at the initial state via $\theta_t$1	$\theta_t$2 in linear MDPs with approximate sampling
Variance-aware contextual bandits	Variance-weighted losses and $\theta_t$3	$\theta_t$4
BART-based nonparametric bandits	Feel-good reweighting of arm-wise BART posterior draws	Minimax-optimal nonparametric regret up to logs for the feel-good variant

In contextual dueling bandits, FGTS.CDB uses two independent posteriors $\theta_t$5 and $\theta_t$6, one for each selected arm. For player $\theta_t$7, the loss is

$\theta_t$8

The independent sampling of $\theta_t$9 and $a_t\in\arg\max_a f(\theta_t,x_t,a)$0 is central to the analysis because it avoids cross terms in the regret decomposition. The main result is $a_t\in\arg\max_a f(\theta_t,x_t,a)$1 regret, matching the lower bound up to logarithmic factors (Li et al., 2024).

In reinforcement learning, the feel-good term is moved to a stage-0 loss at the current initial state,

$a_t\in\arg\max_a f(\theta_t,x_t,a)$2

while the remaining losses are temporal-difference squared losses,

$a_t\in\arg\max_a f(\theta_t,x_t,a)$3

The resulting FG-TS posterior over $a_t\in\arg\max_a f(\theta_t,x_t,a)$4-function parameters is sampled approximately with LMC or ULMC inside least-squares value iteration with approximate sampling exploration. In linear MDPs, the regret bound is

$a_t\in\arg\max_a f(\theta_t,x_t,a)$5

with an additive term that depends on total-variation sampling error (Ishfaq et al., 2024).

The variance-aware extension FGTS-VA modifies both the data-fit weights and the feel-good parameter. With $a_t\in\arg\max_a f(\theta_t,x_t,a)$6 and $a_t\in\arg\max_a f(\theta_t,x_t,a)$7, it sets

$a_t\in\arg\max_a f(\theta_t,x_t,a)$8

and uses the posterior

$a_t\in\arg\max_a f(\theta_t,x_t,a)$9

The corresponding regret bound is

$N$0

and for linear contextual bandits it specializes to $N$1, matching the best variance-aware UCB-type rates cited in that work (Li et al., 3 Nov 2025).

In nonparametric contextual bandits with BART priors, the feel-good BFTS variant is analytically distinct from standard BFTS. The main BFTS algorithm receives a Bayesian regret bound of order $N$2, while the feel-good variant obtains a frequentist nonparametric regret bound

$N$3

which yields

$N$4

under covariate sparsity and Hölder smoothness. The paper states that this is minimax-optimal up to logarithmic factors and presents it as evidence for the “structural suitability of BART priors for non-parametric bandits” (Deng et al., 8 Feb 2026).

5. Approximate sampling, smoothing, and empirical behavior

A substantial later literature treats FG-TS as a sampling problem rather than only a regret-analysis device. The systematic MCMC study benchmarks FG-TS and SFG-TS across eleven real-world and synthetic problems under exact and approximate posterior sampling. The main empirical pattern is conditional: when posterior samples are accurate, such as in linear and logistic bandits with well-tuned MALA or LMC, small feel-good bonuses improve exploration and regret; when sampling noise dominates, especially in neural bandits, the same optimism can amplify approximation error and hurt performance (Anand et al., 21 Jul 2025).

This study also gives a concrete algorithmic view of approximate FG-TS. A generic MCMC-Thompson-sampling template runs an inner chain each round and then acts greedily with the terminal sample. The only change between TS, FG-TS, and SFG-TS is the loss used inside the sampler. The paper evaluates LMC, MALA, HMC, underdamped LMC, preconditioned variants, and SVRG-based variants, and reports that FG-TS “generally outperforms vanilla TS in linear and logistic bandits, but tends to be weaker in neural bandits.” It further recommends FG-TS and its variants as baselines in modern contextual-bandit benchmarks because they are competitive and easy to use (Anand et al., 21 Jul 2025).

The same paper emphasizes hyperparameter sensitivity. Ablations over $N$5 show that small $N$6 can help, while larger values often cause regret spikes. In the linear setting, SFGMALATS with $N$7 achieves cumulative regret comparable to or better than LinUCB and LinTS, but values such as $N$8 and $N$9 often harm performance. In neural contextual bandits, FG-NeuralTS and SFG-NeuralTS often collapse, while vanilla LMC-TS, Neural-$$\sum_{t=1}^T \mathbb{E}[\Delta_t] \ge 0.5\,T(1-1/N)^T,$$0-Greedy, and NeuralUCB are more robust (Anand et al., 21 Jul 2025).

The reinforcement-learning approximation literature shows a different empirical regime. In N-Chain environments and on several Atari 57 games, approximate-sampling FG-TS variants such as FG-LMCDQN and FG-ULMCDQN perform significantly better than strong baselines in hard-exploration problems, and on several Atari games they are reported as better than or on par with other strong baselines from deep RL (Ishfaq et al., 2024). This suggests that the effect of posterior optimism depends strongly on whether exploration difficulty is primarily long-horizon and state-value based, or primarily a posterior-calibration problem in high-dimensional supervised reward models.

The BFTS paper is more cautious. Its feel-good variant is implemented only in a sensitivity study, not as the main empirical method. The reported findings are that the parameter feel_good_eta has large impact, aggressive values can cause normalized regret multipliers of $$\sum_{t=1}^T \mathbb{E}[\Delta_t] \ge 0.5\,T(1-1/N)^T,$$1, and FG-TS “does not consistently outperform BFTS.” The authors conclude that they “do not recommend the feel-good variant as a default in practice” (Deng et al., 8 Feb 2026).

6. Interpretation, misconceptions, and current status

The central conceptual point is that FG-TS is not UCB with randomized scoring. In the contextual-bandit formulation, “there is no extra UCB-style bonus at decision time”; the action rule remains greedy with respect to a sampled model, and the optimism is expressed by reshaping the posterior distribution over models (Anand et al., 21 Jul 2025). The same distinction appears in the foundational paper, where the feel-good term enters the loss, not the action score (Zhang, 2021).

A second point is that minimax-style frequentist guarantees are attached to specific FG-TS constructions, not to Thompson Sampling in general. The original paper proves Bayesian-regret guarantees for standard Thompson Sampling and frequentist guarantees for FG-TS (Zhang, 2021). The BART paper is explicit that the feel-good result “does not constitute a frequentist regret bound for BFTS itself”; instead, it shows that the same BART prior supports minimax-rate learning once paired with a vanishing feel-good perturbation (Deng et al., 8 Feb 2026). The reinforcement-learning approximation framework likewise separates ideal FG-TS regret from the additional regret induced by sampling error through an additive term involving total-variation distance (Ishfaq et al., 2024).

A third point concerns smoothness and implementability. The original $$\sum_{t=1}^T \mathbb{E}[\Delta_t] \ge 0.5\,T(1-1/N)^T,$$2 bonus is nonsmooth, which is why SFG-TS replaces it by a log-sum-exp surrogate for gradient-based MCMC (Anand et al., 21 Jul 2025). In practice, many later FG-TS algorithms are pseudo-posterior methods rather than exact Bayesian procedures: the posterior is a computational and analytical vehicle that blends squared-loss fitting with deliberate optimism. This is explicit in the dueling-bandit construction, which uses tailored pseudo-posteriors, and in the approximate-sampling RL work, which relies on stage-wise MCMC rather than exact joint posterior sampling (Li et al., 2024, Ishfaq et al., 2024).

Across the literature, the present status of FG-TS is therefore mixed but technically clear. In linear contextual bandits and several moderate-dimensional settings, FG-TS and SFG-TS are supported by minimax-style theory and competitive empirical results. In variance-aware and nonparametric settings, the framework has been generalized in a principled way through weighted losses, generalized decoupling coefficients, and BART-based pseudo-posteriors. In neural contextual bandits and some approximate-posterior regimes, however, the optimism term is often reported as fragile. This suggests that the practical utility of FG-TS depends on the fidelity of posterior sampling, the smoothness of the objective, and the extent to which optimism amplifies rather than corrects model miscalibration (Li et al., 3 Nov 2025, Anand et al., 21 Jul 2025).