Momentum-Based Reward Function (MBRF) Overview

• MBRF is a family of reward design strategies that compress temporal information into scalar signals tailored to domain-specific objectives.
• In finance, it employs risk–return distributions and CTS innovations to rank assets, achieving higher returns and lower volatility compared to traditional momentum methods.
• In traffic control and sparse-reward visual RL, MBRF leverages physical momentum and memory discrepancy to improve flow management and exploration efficiency.

Searching arXiv for the cited works and closely related usage of “Momentum-Based Reward Function / momentum reward” to ground the article in the literature. Momentum-Based Reward Function (MBRF) is not a single standardized construct across the arXiv literature. Across published uses, it denotes a scalar reward or ranking signal derived from a notion of momentum, but the underlying meaning of momentum varies by domain: reward–risk momentum in cross-sectional asset selection, physical momentum in traffic signal control, and momentum-memory discrepancy in intrinsic motivation for sparse-reward visual reinforcement learning [(Choi et al., 2014); (Mundane et al., 28 May 2026); (Fang et al., 2022)]. In each case, the reward is not treated as raw payoff alone; it is a function of structured temporal information intended to shape behavior more effectively than conventional return-, queue-, or delay-based objectives.

1. Scope and terminology

The literature shows three distinct technical uses of MBRF-like constructions.

Domain	Reward or score	Meaning of “momentum”
Cross-sectional finance	Sharpe, STAR-ratio, R-ratio, CVaR-based ranking	Momentum portfolio formation using reward–risk scores from past return distributions
Traffic signal control	$R_t = \frac{1}{M}\sum_{i=1}^{M} m_i v_i$	Physical momentum, or average speed in the homogeneous case
Sparse-reward visual RL	$$r_t^I = \left\lVert g_{\theta}(f_{\theta}(s_{t+1})) - g_{\xi}(f_{\xi}(s_{t+1})) \right\rVert_2^2$$	Momentum-updated target network acting as temporal memory

A common misconception is that MBRF refers to a single canonical reward definition. The cited works do not support that interpretation. Instead, the term is domain-specific. In finance, the paper explicitly frames reward–risk ranking as “exactly what you would call a momentum-based reward function (MBRF),” even though the reward is a ranking criterion rather than a reinforcement-learning reward in the narrow sense (Choi et al., 2014). In traffic control, the term is literal and kinematic, with reward equal to average mass-weighted speed over the controlled region (Mundane et al., 28 May 2026). In sparse-reward visual RL, the term emerges through a momentum-updated target network whose discrepancy from the online network defines intrinsic novelty (Fang et al., 2022).

This suggests that MBRF is best understood as a family of reward constructions in which temporal structure is compressed into a scalar signal that directly affects ranking, action selection, or policy optimization.

2. Reward–risk momentum as an MBRF in finance

In the reward–risk momentum framework, standard cross-sectional momentum is modified only at the ranking stage. Assets are not ranked by cumulative past return alone; they are ranked by a reward–risk measure computed from the past return distribution over the last six months. Portfolio construction remains the conventional 6-month lookback, 6-month holding, equal-weighted long/short, dollar-neutral, non-overlapping scheme (Choi et al., 2014).

The paper models returns with an ARMA(1,1)–GARCH(1,1) specification and Classical Tempered Stable (CTS) innovations. The CTS distribution is parameterized as

$$X \sim \mathrm{CTS}(\alpha, C_+, C_-, \lambda_+, \lambda_-, m),$$

with characteristic function

$$\begin{aligned} \phi(u) &= \mathbb{E}(e^{iuX}) \ &= \exp\Big( iu m - iu \Gamma(1-\alpha)(C_+\lambda_+^{1-\alpha} - C_- \lambda_-^{1-\alpha}) \ &\quad + \Gamma(-\alpha)\big[ C_+\big((\lambda_+ - iu)^\alpha - \lambda_+^\alpha\big) + C_-\big((\lambda_- + iu)^\alpha - \lambda_-^\alpha\big) \big] \Big). \end{aligned}$$

The stated rationale is that empirical asset returns exhibit heavy tails, skewness, volatility clustering, and autocorrelation, so CTS innovations are used to obtain more realistic tail-risk quantities than Gaussian assumptions permit.

The ranking metrics include the Sharpe ratio,

$\mathrm{SR} = \frac{E(r - r_f)}{\sigma(r-r_f)},$

CVaR, the STAR-ratio,

$$\mathrm{STAR\text{-}ratio}\big((1-\eta)100\%\big) = \frac{E(r-r_f)}{\mathrm{CVaR}\big((1-\eta)100\%\big)},$$

and the Rachev ratio,

$$\mathrm{R\text{-}ratio}\big((1-\eta)100\%,(1-\zeta)100\%\big) = \frac{\mathrm{CVaR}\big((1-\eta)100\%\big)\ \text{for } (r_f - r)} {\mathrm{CVaR}\big((1-\zeta)100\%\big)\ \text{for } (r - r_f)}.$$

For ratio-based measures, high values form winners and low values form losers; for CVaR, low CVaR forms winners and high CVaR forms losers. Assets are then placed into three baskets in FX, commodities, global indices, and sector ETFs, or into ten deciles in KOSPI 200 and the S&P 500.

The principal empirical result is that reward–risk momentum portfolios, especially the R-ratio$(50\%, 9X\%)$ family, achieve higher average returns with lower volatility, lower VaR/CVaR, and smaller maximum drawdowns than cumulative-return momentum across multiple asset classes. Representative examples include the currency universe, where cumulative-return momentum has mean $0.34\%$ and volatility $2.33\%$, while R-ratio$$r_t^I = \left\lVert g_{\theta}(f_{\theta}(s_{t+1})) - g_{\xi}(f_{\xi}(s_{t+1})) \right\rVert_2^2$$0 has mean $$r_t^I = \left\lVert g_{\theta}(f_{\theta}(s_{t+1})) - g_{\xi}(f_{\xi}(s_{t+1})) \right\rVert_2^2$$1 and volatility $$r_t^I = \left\lVert g_{\theta}(f_{\theta}(s_{t+1})) - g_{\xi}(f_{\xi}(s_{t+1})) \right\rVert_2^2$$2, and R-ratio$$r_t^I = \left\lVert g_{\theta}(f_{\theta}(s_{t+1})) - g_{\xi}(f_{\xi}(s_{t+1})) \right\rVert_2^2$$3 has mean $$r_t^I = \left\lVert g_{\theta}(f_{\theta}(s_{t+1})) - g_{\xi}(f_{\xi}(s_{t+1})) \right\rVert_2^2$$4 and volatility $$r_t^I = \left\lVert g_{\theta}(f_{\theta}(s_{t+1})) - g_{\xi}(f_{\xi}(s_{t+1})) \right\rVert_2^2$$5; maximum drawdown falls from $$r_t^I = \left\lVert g_{\theta}(f_{\theta}(s_{t+1})) - g_{\xi}(f_{\xi}(s_{t+1})) \right\rVert_2^2$$6 to $$r_t^I = \left\lVert g_{\theta}(f_{\theta}(s_{t+1})) - g_{\xi}(f_{\xi}(s_{t+1})) \right\rVert_2^2$$7 for R-ratio$$r_t^I = \left\lVert g_{\theta}(f_{\theta}(s_{t+1})) - g_{\xi}(f_{\xi}(s_{t+1})) \right\rVert_2^2$$8 (Choi et al., 2014). In the S&P 500, R-ratio$$r_t^I = \left\lVert g_{\theta}(f_{\theta}(s_{t+1})) - g_{\xi}(f_{\xi}(s_{t+1})) \right\rVert_2^2$$9 reaches $$X \sim \mathrm{CTS}(\alpha, C_+, C_-, \lambda_+, \lambda_-, m),$$0 per month versus $$X \sim \mathrm{CTS}(\alpha, C_+, C_-, \lambda_+, \lambda_-, m),$$1 for cumulative momentum, with volatility $$X \sim \mathrm{CTS}(\alpha, C_+, C_-, \lambda_+, \lambda_-, m),$$2 versus $$X \sim \mathrm{CTS}(\alpha, C_+, C_-, \lambda_+, \lambda_-, m),$$3, and maximum drawdown declining from about $$X \sim \mathrm{CTS}(\alpha, C_+, C_-, \lambda_+, \lambda_-, m),$$4 to about $$X \sim \mathrm{CTS}(\alpha, C_+, C_-, \lambda_+, \lambda_-, m),$$5–$$X \sim \mathrm{CTS}(\alpha, C_+, C_-, \lambda_+, \lambda_-, m),$$6.

The paper also reports Carhart four-factor regressions for S&P 500 portfolios. Cumulative-return momentum has winner-minus-loser alpha of about $$X \sim \mathrm{CTS}(\alpha, C_+, C_-, \lambda_+, \lambda_-, m),$$7 per month with $$X \sim \mathrm{CTS}(\alpha, C_+, C_-, \lambda_+, \lambda_-, m),$$8, whereas R-ratio portfolios exhibit very low factor loadings, very low $$X \sim \mathrm{CTS}(\alpha, C_+, C_-, \lambda_+, \lambda_-, m),$$9 values such as $$\begin{aligned} \phi(u) &= \mathbb{E}(e^{iuX}) \ &= \exp\Big( iu m - iu \Gamma(1-\alpha)(C_+\lambda_+^{1-\alpha} - C_- \lambda_-^{1-\alpha}) \ &\quad + \Gamma(-\alpha)\big[ C_+\big((\lambda_+ - iu)^\alpha - \lambda_+^\alpha\big) + C_-\big((\lambda_- + iu)^\alpha - \lambda_-^\alpha\big) \big] \Big). \end{aligned}$$0–$$\begin{aligned} \phi(u) &= \mathbb{E}(e^{iuX}) \ &= \exp\Big( iu m - iu \Gamma(1-\alpha)(C_+\lambda_+^{1-\alpha} - C_- \lambda_-^{1-\alpha}) \ &\quad + \Gamma(-\alpha)\big[ C_+\big((\lambda_+ - iu)^\alpha - \lambda_+^\alpha\big) + C_-\big((\lambda_- + iu)^\alpha - \lambda_-^\alpha\big) \big] \Big). \end{aligned}$$1 for winner-minus-loser portfolios, and positive statistically significant alphas of about $$\begin{aligned} \phi(u) &= \mathbb{E}(e^{iuX}) \ &= \exp\Big( iu m - iu \Gamma(1-\alpha)(C_+\lambda_+^{1-\alpha} - C_- \lambda_-^{1-\alpha}) \ &\quad + \Gamma(-\alpha)\big[ C_+\big((\lambda_+ - iu)^\alpha - \lambda_+^\alpha\big) + C_-\big((\lambda_- + iu)^\alpha - \lambda_-^\alpha\big) \big] \Big). \end{aligned}$$2–$$\begin{aligned} \phi(u) &= \mathbb{E}(e^{iuX}) \ &= \exp\Big( iu m - iu \Gamma(1-\alpha)(C_+\lambda_+^{1-\alpha} - C_- \lambda_-^{1-\alpha}) \ &\quad + \Gamma(-\alpha)\big[ C_+\big((\lambda_+ - iu)^\alpha - \lambda_+^\alpha\big) + C_-\big((\lambda_- + iu)^\alpha - \lambda_-^\alpha\big) \big] \Big). \end{aligned}$$3 per month for R-ratio$$\begin{aligned} \phi(u) &= \mathbb{E}(e^{iuX}) \ &= \exp\Big( iu m - iu \Gamma(1-\alpha)(C_+\lambda_+^{1-\alpha} - C_- \lambda_-^{1-\alpha}) \ &\quad + \Gamma(-\alpha)\big[ C_+\big((\lambda_+ - iu)^\alpha - \lambda_+^\alpha\big) + C_-\big((\lambda_- + iu)^\alpha - \lambda_-^\alpha\big) \big] \Big). \end{aligned}$$4 and R-ratio$$\begin{aligned} \phi(u) &= \mathbb{E}(e^{iuX}) \ &= \exp\Big( iu m - iu \Gamma(1-\alpha)(C_+\lambda_+^{1-\alpha} - C_- \lambda_-^{1-\alpha}) \ &\quad + \Gamma(-\alpha)\big[ C_+\big((\lambda_+ - iu)^\alpha - \lambda_+^\alpha\big) + C_-\big((\lambda_- + iu)^\alpha - \lambda_-^\alpha\big) \big] \Big). \end{aligned}$$5 (Choi et al., 2014). By contrast, Sharpe and STAR portfolios are described as more factor-like, and CVaR portfolios as factor-driven but with sizeable intercepts. A plausible implication is that, in this setting, the choice of reward–risk functional shapes not only performance and tail exposure but also the degree of factor neutrality.

3. Physical momentum as a reward in traffic signal control

In low-emission traffic signal control, MBRF is defined exactly as

$$\begin{aligned} \phi(u) &= \mathbb{E}(e^{iuX}) \ &= \exp\Big( iu m - iu \Gamma(1-\alpha)(C_+\lambda_+^{1-\alpha} - C_- \lambda_-^{1-\alpha}) \ &\quad + \Gamma(-\alpha)\big[ C_+\big((\lambda_+ - iu)^\alpha - \lambda_+^\alpha\big) + C_-\big((\lambda_- + iu)^\alpha - \lambda_-^\alpha\big) \big] \Big). \end{aligned}$$6

where $$\begin{aligned} \phi(u) &= \mathbb{E}(e^{iuX}) \ &= \exp\Big( iu m - iu \Gamma(1-\alpha)(C_+\lambda_+^{1-\alpha} - C_- \lambda_-^{1-\alpha}) \ &\quad + \Gamma(-\alpha)\big[ C_+\big((\lambda_+ - iu)^\alpha - \lambda_+^\alpha\big) + C_-\big((\lambda_- + iu)^\alpha - \lambda_-^\alpha\big) \big] \Big). \end{aligned}$$7 is the number of vehicles currently in the intersection region, $$\begin{aligned} \phi(u) &= \mathbb{E}(e^{iuX}) \ &= \exp\Big( iu m - iu \Gamma(1-\alpha)(C_+\lambda_+^{1-\alpha} - C_- \lambda_-^{1-\alpha}) \ &\quad + \Gamma(-\alpha)\big[ C_+\big((\lambda_+ - iu)^\alpha - \lambda_+^\alpha\big) + C_-\big((\lambda_- + iu)^\alpha - \lambda_-^\alpha\big) \big] \Big). \end{aligned}$$8 is the mass of vehicle $$\begin{aligned} \phi(u) &= \mathbb{E}(e^{iuX}) \ &= \exp\Big( iu m - iu \Gamma(1-\alpha)(C_+\lambda_+^{1-\alpha} - C_- \lambda_-^{1-\alpha}) \ &\quad + \Gamma(-\alpha)\big[ C_+\big((\lambda_+ - iu)^\alpha - \lambda_+^\alpha\big) + C_-\big((\lambda_- + iu)^\alpha - \lambda_-^\alpha\big) \big] \Big). \end{aligned}$$9, and $\mathrm{SR} = \frac{E(r - r_f)}{\sigma(r-r_f)},$0 is its instantaneous speed (Mundane et al., 28 May 2026). In homogeneous experiments, the authors set $\mathrm{SR} = \frac{E(r - r_f)}{\sigma(r-r_f)},$1 for all vehicles, so the reward reduces to

$\mathrm{SR} = \frac{E(r - r_f)}{\sigma(r-r_f)},$2

that is, average speed. In heterogeneous experiments, representative masses are passenger car $\mathrm{SR} = \frac{E(r - r_f)}{\sigma(r-r_f)},$3, truck $\mathrm{SR} = \frac{E(r - r_f)}{\sigma(r-r_f)},$4, bus $\mathrm{SR} = \frac{E(r - r_f)}{\sigma(r-r_f)},$5, and motorcycle $\mathrm{SR} = \frac{E(r - r_f)}{\sigma(r-r_f)},$6, with scaling relative to a standard car mass of $\mathrm{SR} = \frac{E(r - r_f)}{\sigma(r-r_f)},$7.

The paper’s design rationale is that waiting-time and queue-length rewards are negative penalties on stock variables, whereas MBRF is a positive reward on a flow quantity. Waiting-time and queue-based rewards may be relatively flat in severe congestion and can induce flickering phase changes; MBRF reacts continuously to speed changes and therefore provides denser feedback. The intended behavioral effect is to maintain motion, reduce stop-and-go behavior, and improve the throughput–emission trade-off without constructing a hand-tuned multi-term reward.

The DRL implementation uses a DQN with a multilayer perceptron of two hidden layers with 64 neurons each and ReLU activation, experience replay, and a target network updated every 500 steps. Hyperparameters include learning rate $\mathrm{SR} = \frac{E(r - r_f)}{\sigma(r-r_f)},$8, discount factor $\mathrm{SR} = \frac{E(r - r_f)}{\sigma(r-r_f)},$9, $$\mathrm{STAR\text{-}ratio}\big((1-\eta)100\%\big) = \frac{E(r-r_f)}{\mathrm{CVaR}\big((1-\eta)100\%\big)},$$0-greedy exploration decaying from $$\mathrm{STAR\text{-}ratio}\big((1-\eta)100\%\big) = \frac{E(r-r_f)}{\mathrm{CVaR}\big((1-\eta)100\%\big)},$$1 to $$\mathrm{STAR\text{-}ratio}\big((1-\eta)100\%\big) = \frac{E(r-r_f)}{\mathrm{CVaR}\big((1-\eta)100\%\big)},$$2, episode duration $$\mathrm{STAR\text{-}ratio}\big((1-\eta)100\%\big) = \frac{E(r-r_f)}{\mathrm{CVaR}\big((1-\eta)100\%\big)},$$3, and evaluation every 10,000 training steps. The state is

$$\mathrm{STAR\text{-}ratio}\big((1-\eta)100\%\big) = \frac{E(r-r_f)}{\mathrm{CVaR}\big((1-\eta)100\%\big)},$$4

with current phase, minimum-green indicator, lane densities, and lane queue lengths; the action space is $$\mathrm{STAR\text{-}ratio}\big((1-\eta)100\%\big) = \frac{E(r-r_f)}{\mathrm{CVaR}\big((1-\eta)100\%\big)},$$5 for North–South green or East–West green at a control interval $$\mathrm{STAR\text{-}ratio}\big((1-\eta)100\%\big) = \frac{E(r-r_f)}{\mathrm{CVaR}\big((1-\eta)100\%\big)},$$6, with SUMO inserting a $$\mathrm{STAR\text{-}ratio}\big((1-\eta)100\%\big) = \frac{E(r-r_f)}{\mathrm{CVaR}\big((1-\eta)100\%\big)},$$7 yellow on switches (Mundane et al., 28 May 2026).

The reported homogeneous-setting results show that MBRF achieves the highest throughput and lowest CO$$\mathrm{STAR\text{-}ratio}\big((1-\eta)100\%\big) = \frac{E(r-r_f)}{\mathrm{CVaR}\big((1-\eta)100\%\big)},$$8 emissions among the listed controllers. Specifically, MBRF yields waiting time $$\mathrm{STAR\text{-}ratio}\big((1-\eta)100\%\big) = \frac{E(r-r_f)}{\mathrm{CVaR}\big((1-\eta)100\%\big)},$$9, queue length $$\mathrm{R\text{-}ratio}\big((1-\eta)100\%,(1-\zeta)100\%\big) = \frac{\mathrm{CVaR}\big((1-\eta)100\%\big)\ \text{for } (r_f - r)} {\mathrm{CVaR}\big((1-\zeta)100\%\big)\ \text{for } (r - r_f)}.$$0, throughput $$\mathrm{R\text{-}ratio}\big((1-\eta)100\%,(1-\zeta)100\%\big) = \frac{\mathrm{CVaR}\big((1-\eta)100\%\big)\ \text{for } (r_f - r)} {\mathrm{CVaR}\big((1-\zeta)100\%\big)\ \text{for } (r - r_f)}.$$1, travel time $$\mathrm{R\text{-}ratio}\big((1-\eta)100\%,(1-\zeta)100\%\big) = \frac{\mathrm{CVaR}\big((1-\eta)100\%\big)\ \text{for } (r_f - r)} {\mathrm{CVaR}\big((1-\zeta)100\%\big)\ \text{for } (r - r_f)}.$$2, and CO$$\mathrm{R\text{-}ratio}\big((1-\eta)100\%,(1-\zeta)100\%\big) = \frac{\mathrm{CVaR}\big((1-\eta)100\%\big)\ \text{for } (r_f - r)} {\mathrm{CVaR}\big((1-\zeta)100\%\big)\ \text{for } (r - r_f)}.$$3 emissions $$\mathrm{R\text{-}ratio}\big((1-\eta)100\%,(1-\zeta)100\%\big) = \frac{\mathrm{CVaR}\big((1-\eta)100\%\big)\ \text{for } (r_f - r)} {\mathrm{CVaR}\big((1-\zeta)100\%\big)\ \text{for } (r - r_f)}.$$4. The nearest DRL baselines are DQN-Diff with throughput $$\mathrm{R\text{-}ratio}\big((1-\eta)100\%,(1-\zeta)100\%\big) = \frac{\mathrm{CVaR}\big((1-\eta)100\%\big)\ \text{for } (r_f - r)} {\mathrm{CVaR}\big((1-\zeta)100\%\big)\ \text{for } (r - r_f)}.$$5 and CO$$\mathrm{R\text{-}ratio}\big((1-\eta)100\%,(1-\zeta)100\%\big) = \frac{\mathrm{CVaR}\big((1-\eta)100\%\big)\ \text{for } (r_f - r)} {\mathrm{CVaR}\big((1-\zeta)100\%\big)\ \text{for } (r - r_f)}.$$6 $$\mathrm{R\text{-}ratio}\big((1-\eta)100\%,(1-\zeta)100\%\big) = \frac{\mathrm{CVaR}\big((1-\eta)100\%\big)\ \text{for } (r_f - r)} {\mathrm{CVaR}\big((1-\zeta)100\%\big)\ \text{for } (r - r_f)}.$$7, and DQN-Queue with throughput $$\mathrm{R\text{-}ratio}\big((1-\eta)100\%,(1-\zeta)100\%\big) = \frac{\mathrm{CVaR}\big((1-\eta)100\%\big)\ \text{for } (r_f - r)} {\mathrm{CVaR}\big((1-\zeta)100\%\big)\ \text{for } (r - r_f)}.$$8 and CO$$\mathrm{R\text{-}ratio}\big((1-\eta)100\%,(1-\zeta)100\%\big) = \frac{\mathrm{CVaR}\big((1-\eta)100\%\big)\ \text{for } (r_f - r)} {\mathrm{CVaR}\big((1-\zeta)100\%\big)\ \text{for } (r - r_f)}.$$9 $(50\%, 9X\%)$0. Classical controllers perform markedly worse: Max Pressure gives throughput $(50\%, 9X\%)$1 and LQF gives $(50\%, 9X\%)$2 (Mundane et al., 28 May 2026).

The paper further attributes improved stability to the smoothness of the reward signal. Throughput variance is lower under MBRF than under DQN-Diff or DQN-Queue, and CO$(50\%, 9X\%)$3 variance is also smaller. In the heterogeneous setting, MBRF again attains the highest throughput, $(50\%, 9X\%)$4, and near-lowest emissions, $(50\%, 9X\%)$5, slightly better than DQN-Diff at $(50\%, 9X\%)$6. This supports the claim that mass-weighted motion can serve as a single-term reward that indirectly improves emissions and flow quality.

4. Momentum memory as an intrinsic reward in sparse-reward visual RL

In sparse-reward visual navigation, the Momentum Memory Intrinsic Reward (MMIR) is an intrinsic MBRF defined from the discrepancy between an online representation network and a target representation network updated by exponential moving average (EMA) (Fang et al., 2022). The target parameters obey

$(50\%, 9X\%)$7

with $(50\%, 9X\%)$8 in the reported experiments. The online and target representations are

$(50\%, 9X\%)$9

and the intrinsic reward is

$0.34\%$0

Large values indicate novelty; small values indicate familiarity.

This reward is embedded in the IAMMIR framework, which combines self-supervised representation learning, image augmentation, momentum memory, and PPO. The dense reward is

$0.34\%$1

with $0.34\%$2. The representation-learning component is Image-Augmented Forward Dynamics Representation (IAFDR). Raw states are stacks of four grayscale images in $0.34\%$3, augmented by random shifts and random brightness transformations. A forward dynamics head predicts the next latent state,

$0.34\%$4

and is trained with cosine similarity loss

$0.34\%$5

The overall objective is

$0.34\%$6

with $0.34\%$7.

The benchmark tasks are VizdoomMyWayHome in VerySparse and Dense variants, and VizdoomFlytrap. Episodes terminate on success or after 2100 steps in MyWayHome, and run up to 10,000 steps in Flytrap. PPO alone fails in the hardest sparse settings, whereas PPO with intrinsic rewards can solve them. The reported result is that IAMMIR reaches 100% success at least $0.34\%$8 faster than ICM in VerySparse, $0.34\%$9 faster in Flytrap, and at least $2.33\%$0 faster than ECO in both VerySparse and Flytrap (Fang et al., 2022).

A central implementation finding is that MMIR should be computed on projector outputs rather than encoder outputs. The paper reports that projector-based MMIR decreases monotonically during training, which is interpreted as increasing state familiarity, whereas encoder-based MMIR can increase, indicating representation hijacking. The ablation study further states that removing MMIR causes failure on the VerySparse task, while removing IAFDR causes a large drop in learning speed. In this formulation, momentum refers neither to physical motion nor to market trend, but to the momentum-updated target network that provides a temporally smoothed memory of past representations.

5. Comparative structure across domains

The three formulations are technically different, but they share a recognizable structural pattern.

Work	Input to reward	Operational effect
Reward–risk momentum	Estimated conditional return distribution over a 6-month window	Determines winner/loser ranking in long–short momentum portfolios
Traffic MBRF	Per-step masses and instantaneous speeds in the intersection region	Guides phase selection toward sustained flow
MMIR	Online–target representation discrepancy at the next state	Densifies sparse reward and drives exploration

This suggests a domain-general interpretation of MBRF as a scalar functional of temporally organized information rather than an immediate raw outcome. In the finance setting, the scalar is derived from the full conditional return distribution, with emphasis on downside-tail asymmetry and CTS-based risk estimation. In traffic control, it is derived from instantaneous kinematics aggregated over the control region. In sparse-reward visual RL, it is derived from disagreement between current and momentum-smoothed historical representations.

A second commonality is that all three formulations alter behavior by changing what is considered salient. Reward–risk ranking replaces cumulative return with distribution-aware scores. Traffic MBRF replaces queue or delay penalties with motion-centered reward. MMIR replaces sparse terminal feedback alone with a novelty signal that decays as familiarity increases. A plausible implication is that the core contribution of an MBRF is often not algorithmic novelty in the optimizer itself, but a redefinition of the objective signal presented to an otherwise standard portfolio rule or RL algorithm.

6. Limitations, assumptions, and research directions

The cited works also make clear that MBRF design is sensitive to modeling assumptions. In finance, the reward–risk framework does not model transaction costs or liquidity, uses a static 6/6 horizon, and depends on ARMA–GARCH–CTS specification; the paper explicitly notes model risk, possible parameter instability, and the absence of turnover constraints or position limits beyond equal weighting (Choi et al., 2014). In traffic control, the evaluation is limited to a single isolated four-arm intersection with two phases, full observability, fixed phase structure, no explicit safety metric, and a single DRL algorithm, DQN (Mundane et al., 28 May 2026). In sparse-reward visual RL, MMIR is sensitive to projector design and to hyperparameters such as $2.33\%$1, $2.33\%$2, and $2.33\%$3, and the paper identifies pathologies when intrinsic reward is computed directly on encoder outputs (Fang et al., 2022).

The proposed extensions are correspondingly domain-specific. The finance paper discusses alternative tempered stable or generalized hyperbolic families, dynamic parameter updating, regime switching, cost-aware reward formulations, and multi-objective RL-style penalties on factor exposure (Choi et al., 2014). The traffic paper identifies extensions to multi-intersection networks, PPO and actor–critic methods, partial observability, communication latency, and more complex phase structures (Mundane et al., 28 May 2026). The visual RL paper points to other RL algorithms such as DQN, SAC, and A3C, alternative latent-space distance metrics, and continuous-control applications (Fang et al., 2022).

Taken together, these limitations reinforce a final point: MBRF is not defined by a single formula, but by a design principle. The reward is constructed to encode a temporally meaningful notion of progress or quality—tail-aware performance, sustained vehicular motion, or novelty relative to a momentum-smoothed memory—rather than relying on a simpler proxy that may be sparse, noisy, or misaligned with the target behavior.