Loyalty Points Bandits

• Loyalty Points Bandits are dynamic models that integrate fidelity rewards into multi-armed bandit frameworks, balancing revenue, fairness, and user heterogeneity.
• They employ various reward functions – increasing, decreasing, and periodic – to mimic real-world CRM programs and address exploration-exploitation trade-offs.
• Recent algorithmic innovations, such as Stable-Greedy and UCB-LOY, achieve sublinear regret while ensuring temporal fairness and robustness to model misspecification.

Loyalty Points Bandits integrate dynamic incentives for repeated engagement within online decision-making models, primarily multi-armed bandits (MAB), to formalize and optimize customer rewards programs, fidelity mechanisms, and engagement strategies. The core challenge is balancing effective revenue maximization, customer heterogeneity, temporal fairness, and the inherent uncertainty in customer responses. This field synthesizes stochastic and adversarial bandit algorithms, latent-state modeling, and combinatorial optimization, giving rise to numerous variants including the loyalty-points model, subscription-based fidelity rewards, and combinatorial matching with dynamic agent preferences.

1. Mathematical Foundations of Loyalty Points Bandits

The canonical "loyalty points bandit" model augments each arm of a $K$-armed bandit with a fidelity or loyalty-points reward conditional on the past play count or recent engagement pattern. Formally, at round $t$, arm $j$ yields a base reward $X_{t,j}\in[0,1]$, and the player receives a fidelity reward $f_j(N_{t-1,j})$, where $N_{t-1,j}$ is the cumulative number of times arm $j$ was selected up to time $t-1$ (Lugosi et al., 2021). The cumulative expected reward of a sequence $j=(j_1,\dots,j_T)$ under a policy $\pi$ is: $t$0 Variants include:

• Increasing $t$1: Rewards (e.g., points per purchase) strictly improve as loyalty increases.
• Decreasing $t$2: Rewards decay with repeated use (“rotting,” e.g., diminishing returns for repeated offers).
• Coupon (periodic) $t$3: Step-like bonuses awarded at fixed intervals (e.g., after every $t$4 plays).

In resource-constrained, personalized settings, the arms formalize user-offer pairs and the system incorporates Markovian latent states $t$5 representing user preferences, which may evolve upon receiving offers according to a transition kernel $t$6 (Fiez et al., 2018).

2. Fairness, Personalization, and the Price of Fairness

A major axis differentiates between personalized and individually fair loyalty programs. In the points-based rewards program setting (Hssaine et al., 4 Jun 2025):

• The personalized approach allows distinct redemption thresholds $t$7 for customers of type $t$8; revenue is maximized over the vector $t$9.
• The individually fair program uses a single universal $j$0 applied to the entire population.

The price of fairness (PoF) quantifies the worst-case revenue loss incurred by enforcing a universal threshold: $j$1 where $j$2 is optimal revenue under personalization and $j$3 is under universal thresholds. The sharp upper bound is $j$4 for general $j$5 types, and $j$6 for $j$7 (Hssaine et al., 4 Jun 2025). Empirically, $j$8 is often $j$9–$X_{t,j}\in[0,1]$0 in realistic scenarios, indicating limited marginal benefit from personalization except in highly heterogeneous populations.

3. Learning Algorithms for Loyalty Points Bandits

Bandit learning under demand and feedback uncertainty is complicated by the need for temporal fairness (avoiding devaluation of earned points due to threshold updates) and exploration-exploitation trade-offs. Key algorithms include:

• Stable-Greedy (Points-Bandits):
  • Partition $X_{t,j}\in[0,1]$1 periods into $X_{t,j}\in[0,1]$2 epochs.
  • In each epoch, use MLE fits to the GLM user-response model to set optimal threshold $X_{t,j}\in[0,1]$3 as $X_{t,j}\in[0,1]$4.
  • Adjust $X_{t,j}\in[0,1]$5 only $X_{t,j}\in[0,1]$6 times for minimal disruption and optimal regret $X_{t,j}\in[0,1]$7 (Hssaine et al., 4 Jun 2025).
• Fair-Greedy:
  • Always sets $X_{t,j}\in[0,1]$8 nonincreasingly (i.e., thresholds only decrease), further improving temporal fairness at the expense of a constant-factor increase in regret (still $X_{t,j}\in[0,1]$9).
  • Terminate the loyalty program if estimated no-loyalty revenue exceeds the best threshold by a confidence-adjusted margin (Hssaine et al., 4 Jun 2025).
• UCB-LOY (Stochastic Loyalty-Points Bandits):
  • For nondecreasing $f_j(N_{t-1,j})$0, a UCB algorithm on the “augmented” rewards ($f_j(N_{t-1,j})$1) ensures optimality; switching arms is suboptimal due to forfeited future loyalty rewards.
  • Achieves worst-case regret $f_j(N_{t-1,j})$2 (Lugosi et al., 2021).
• EXP4-type algorithms (Adversarial/Rotting Settings):
  • To handle concave or decaying $f_j(N_{t-1,j})$3, exploit a covering argument over the simplex and run a forecaster with importance-weighted losses and explicit convexity of the fidelity term.
  • Guarantees $f_j(N_{t-1,j})$4 mean or weak regret when strong regret is necessarily linear (Lugosi et al., 2021).
• MG-EUCB (Combinatorial Bandit with User Dynamics):
  • For personalized offers in combinatorial settings with Markovian user dynamics, a Markov-mixing-corrected UCB index is employed within greedy matchings subject to resource constraints.
  • Achieves gap-dependent regret $f_j(N_{t-1,j})$5 (Fiez et al., 2018).

4. Regret Definitions and Tradeoffs

Loyalty-points bandits require nonstandard regret notions:

• Pseudo-regret: Difference versus the best fixed sequence or pattern of plays, accounting for fidelity rewards (Lugosi et al., 2021).
• Strong/weak/mean regret: Especially relevant for adversarial environments, where the optimal sequence is context-dependent and often not single-armed. Weak and mean regret relaxations are necessary to admit sublinear guarantees when strong regret is $f_j(N_{t-1,j})$6 (Lugosi et al., 2021).

Switching policies and dynamic thresholding, especially in points-accumulation schemes, introduce temporal fairness concerns since increasing thresholds mid-sequence (i.e., devaluing users’ progress) is penalized by users and can invalidate the fairness criterion. Monotonic policies that only decrease thresholds (or keep them fixed) mitigate this effect, as in the Fair-Greedy algorithm (Hssaine et al., 4 Jun 2025).

5. Empirical and Numerical Insights

Simulation studies in (Hssaine et al., 4 Jun 2025) and (Fiez et al., 2018) confirm key theoretical findings:

• Universal thresholds nearly match personalized policies in average-case performance, with empirical $f_j(N_{t-1,j})$7–$f_j(N_{t-1,j})$8 even for $f_j(N_{t-1,j})$9.
• Both Stable-Greedy and Fair-Greedy algorithms achieve sublinear regret, with the fairer algorithm changing the threshold significantly fewer times and never increasing it, thereby preserving fairness at limited cost.
• Robustness to model misspecification is demonstrated; fitting a linear instead of exponential or logit user response still achieves $N_{t-1,j}$0–$N_{t-1,j}$1 of optimal revenue after moderate time horizons (Hssaine et al., 4 Jun 2025).
• In combinatorial loyalty-point matching with dynamic user preferences, Markov-corrected UCB policies adaptively learn user models and match omniscient greedy performance up to a logarithmic factor, provided latent-state mixing is accounted for (Fiez et al., 2018).

6. Special Models, Phase Transitions, and Implementation Guidelines

A range of fidelity reward structures—monotonic, concave, periodic—induce qualitatively different learning regimes and regret bounds (Lugosi et al., 2021):

• Increasing-fidelity: Stochastic setting admits efficient UCB-type learning; adversarial setting yields linear regret unless $N_{t-1,j}$2 is nearly constant.
• Decreasing-fidelity ("rotting"): Both stochastic and adversarial settings permit sublinear mean/weak regret via tailored explore-exploit strategies.
• Coupon rewards: Periodic bonuses are tractable via batched or shifted-reward bandit algorithms, achieving $N_{t-1,j}$3 regret.

Practical recommendations:

• Enforce monotonic thresholding (only decrease thresholds) to maintain user trust and avoid point devaluation (Hssaine et al., 4 Jun 2025).
• Default to a universal threshold unless customer heterogeneity is extreme; personalization rarely yields substantial marginal gains.
• In combinatorial schemes, always incorporate Markov-mixing corrections into confidence intervals and use covering initialization to avoid estimation bias (Fiez et al., 2018).
• Design $N_{t-1,j}$4 to be concave or step-like for learnability in adversarial/environmentally nonstationary settings (Lugosi et al., 2021).

7. Connections, Implications, and Future Directions

Loyalty Points Bandits synthesize technical threads from stochastic/online learning, incentive design, fairness in sequential decision-making, and practical CRM systems. The rigorous quantification of fairness–efficiency trade-offs, sublinear-regret algorithm construction under resource and temporal constraints, and the interplay of combinatorial optimization with latent-state user dynamics mark ongoing research frontiers.

A plausible implication is the growing need for interpretable, minimally disruptive experimentation protocols in loyalty-based CRM and e-commerce platforms: subtle choices in threshold scheduling and bandit algorithm design may have outsized effects on both fairness perceptions and business outcomes. Furthermore, the emerging findings on limited personalization value underlines the importance of robust, globally-fair mechanisms over segmentation-heavy strategies, except where observational heterogeneity is extreme.

Key references:

• "Learning Fair And Effective Points-Based Rewards Programs" (Hssaine et al., 4 Jun 2025)
• "Bandit problems with fidelity rewards" (Lugosi et al., 2021)
• "Combinatorial Bandits for Incentivizing Agents with Dynamic Preferences" (Fiez et al., 2018)