Uncertainty-Modulated Gain Dynamics

• Uncertainty-modulated gain dynamics are mechanisms that adjust feedback strength or learning updates based on online estimates of uncertainty, enabling rapid stabilization alongside gradual model refinement.
• This framework unifies diverse domains, including sensorimotor adaptation, continual learning via neuromodulation, entropy-modulated policy gradients in LLMs, and adaptive Kalman filtering in Bayesian inference.
• By differentiating reactive and predictive gains through signals like internal-model error, entropy, and volatility, these dynamics drive robust control and efficient adaptation across complex, real-world tasks.

Uncertainty-modulated gain dynamics denotes a family of mechanisms in which an internal estimate of uncertainty modulates the strength of feedback, plasticity, or policy updates. In the cited literature, the modulated quantity is variously a visuomotor feedback gain, a neuronal gain factor, a policy-gradient scaling term, or a Kalman gain; the uncertainty signal is variously internal-model uncertainty $\sigma$, softmax entropy $H(y)$, step entropy $H_t$, volatility $\sigma_v^2$, or stochasticity $\sigma_s^2$ (Franklin et al., 2020, Rodriguez-Garcia et al., 18 Jul 2025, Wang et al., 11 Sep 2025, Piray, 19 May 2026). A recurring theme is the separation of rapid stabilization or adaptation from slower model refinement, which links robust-control, Bayesian, and neuromodulatory accounts of learning.

1. Cross-domain structure

The same architectural motif appears in several otherwise distinct research programs.

Domain	Uncertainty signal	Gain-modulated quantity
Sensorimotor adaptation	internal-model uncertainty $\sigma$; large kinematic errors	$G = G_0 + G_r(\sigma) + G_p(dynamics)$
Continual learning	softmax output entropy $H(y)$	$g_i(t)$ in $W_{ij}(t)=g_i(t)\,w_{ij}(t)$
Long-horizon LLM agents	step-wise entropy $H(y)$0	$H(y)$1 in entropy-modulated policy gradients
Gaussian state-space bandits	volatility $H(y)$2; stochasticity $H(y)$3	Kalman gain $H(y)$4

The shared pattern is not a single algorithm but a common control law: uncertainty is estimated online, mapped into a gain variable, and then applied to a feedback or learning channel. This suggests a general schema in which gain is neither fixed nor treated as a mere hyperparameter, but becomes a state-dependent variable coupled to inference or adaptation (Franklin et al., 2020, Rodriguez-Garcia et al., 18 Jul 2025, Wang et al., 11 Sep 2025, Piray, 19 May 2026).

A second commonality is the distinction between transient and steady-state effects. In motor adaptation, reactive gains arise immediately under large model mismatch, whereas predictive gains emerge only after learning. In noradrenergic gain modulation, phasic excursions sit on top of a tonic baseline. In policy gradients, uncertain steps are downweighted relative to confident ones, but the batch-average scaling is constrained so that the mean gain remains $H(y)$5. In Kalman filtering, the learning rate itself is the time-varying gain, with a stationary limit under fixed volatility and stochasticity.

2. Formal mechanisms

In the noradrenergic-inspired continual-learning formulation, each neuron $H(y)$6 has an effective synaptic weight

$H(y)$7

Here $H(y)$8 is the underlying “slow” synaptic weight and $H(y)$9 is a scalar gain capturing neuromodulatory state. The weight update is

$H_t$0

and the gain update is

$H_t$1

with $H_t$2. The tonic baseline gain $H_t$3 carries slow, stable learning, whereas phasic excursions driven by entropy provide a transient fast channel (Rodriguez-Garcia et al., 18 Jul 2025).

In sensorimotor adaptation, total feedback gain is decomposed as

$H_t$4

$H_t$5 is the baseline null-field gain, $H_t$6 is a reactive gain that depends on internal-model uncertainty $H_t$7, and $H_t$8 is a predictive gain learned for the external dynamics. The reactive scaling hypothesis is

$H_t$9

and after many trials the gain approaches

$\sigma_v^2$0

where $\sigma_v^2$1 is the steady-state gain matching the novel dynamics (Franklin et al., 2020).

In Entropy-Modulated Policy Gradients, the step-wise uncertainty measure is token-level entropy averaged over a complete “reason-then-act” step,

$\sigma_v^2$2

After mini-batch min–max normalization, each step is reweighted by

$\sigma_v^2$3

so that confident steps with below-batch-mean entropy receive $\sigma_v^2$4 and uncertain steps receive $\sigma_v^2$5. The modulated advantage is

$\sigma_v^2$6

where the future-clarity bonus encourages transitions into lower-entropy next steps (Wang et al., 11 Sep 2025).

3. Sensorimotor adaptation and dual feedback gains

The motor-control study of feedback gain modulation examined reaching movements under abrupt or gradual curl force-field adaptation. Participants grasped a robotic manipulandum and reached in the horizontal plane. The abrupt condition introduced a full-strength curl force field, $\sigma_v^2$7, from the first exposure trial, whereas the gradual condition ramped the field linearly from $\sigma_v^2$8 to full strength over $\sigma_v^2$9 trials and then held it constant. Probe trials constrained the hand to a mechanical channel; at $\sigma_s^2$0 movement extent the visual cursor was jumped laterally by $\sigma_s^2$1 cm for $\sigma_s^2$2 ms, and forces against the channel wall were recorded (Franklin et al., 2020).

The reported dissociation between abrupt and gradual introduction is central. Abrupt introduction elicited large initial increases in maximum perpendicular error, muscle co-contraction, and visuomotor feedback gains. Gradual introduction showed little initial change in these measures despite evidence of adaptation, including $\sigma_s^2$3 force compensation. EMG integrated from $\sigma_s^2$4 ms to $\sigma_s^2$5 ms showed large initial co-contraction across shoulder, biarticular, and elbow muscles in the abrupt condition, followed by decline as predictive compensation built. In the gradual condition, co-contraction was minimal early and muscle activation gradually increased until final EMG levels matched the abrupt condition.

Visuomotor feedback gains were computed as channel-force divided by perturbation size and averaged over an early interval of $\sigma_s^2$6–$\sigma_s^2$7 ms and a late interval of $\sigma_s^2$8–$\sigma_s^2$9 ms. Under abrupt introduction, the early-interval gain on the first exposure trial peaked at approximately $\sigma$0 of null-field and the late-interval gain at approximately $\sigma$1. These reactive elevations then declined slightly over tens of trials as internal-model uncertainty decreased. In both abrupt and gradual conditions, the final plateau converged to approximately $\sigma$2 above null in the early interval and approximately $\sigma$3 above null in the late interval.

The study interprets these data as evidence for two distinct processes. The initial gain increase is not part of the adaptation process; it is an automatic reactive response to internal-model uncertainty. The final elevated gain level is a predictive tuning of the feedback controller to the external dynamics as part of motor-memory adaptation. This dual-process account unifies previous experimental reports by separating fast uncertainty-driven stabilization from slower adaptation-driven retuning.

4. Noradrenergic-inspired gain modulation in continual learning

In continual learning, uncertainty-modulated gain dynamics has been proposed as a mechanism for attenuating the “stability gap,” defined as the transient drop in performance on mastered tasks when assimilating new ones. The proposed mechanism is explicitly inspired by locus coeruleus mediated noradrenergic bursts, which transiently enhance neuronal gain under uncertainty to facilitate sensory assimilation. Tonic LC firing is mapped to a baseline gain $\sigma$4 and slow, stable learning; unexpected environmental change is mapped to phasic gain increases $\sigma$5; output entropy $\sigma$6 approximates “unexpected uncertainty”; and the decay parameter $\sigma$7 models the return to baseline firing after a burst (Rodriguez-Garcia et al., 18 Jul 2025).

The evaluation considered both class-incremental and domain-incremental settings under joint training. Class-incremental experiments used Split MNIST with $\sigma$8 tasks and a $\sigma$9-unit MLP, and Split CIFAR-10 with $G = G_0 + G_r(\sigma) + G_p(dynamics)$0 tasks and a Slim ResNet-18 with gain only on the $G = G_0 + G_r(\sigma) + G_p(dynamics)$1-unit output layer. Domain-incremental experiments used Permuted MNIST with $G = G_0 + G_r(\sigma) + G_p(dynamics)$2 random-permutation tasks, Rotated MNIST with $G = G_0 + G_r(\sigma) + G_p(dynamics)$3 rotations $G = G_0 + G_r(\sigma) + G_p(dynamics)$4, and Domain CIFAR-100 with $G = G_0 + G_r(\sigma) + G_p(dynamics)$5 superclasses. Continuous evaluation used avg-ACC, avg-min-ACC, WC-ACC, and the stability gap

$G = G_0 + G_r(\sigma) + G_p(dynamics)$6

with $G = G_0 + G_r(\sigma) + G_p(dynamics)$7.

Quantitatively, the reported mean $G = G_0 + G_r(\sigma) + G_p(dynamics)$8 SE over $G = G_0 + G_r(\sigma) + G_p(dynamics)$9 runs were as follows. On Permuted MNIST, NGM achieved avg-ACC $H(y)$0, avg-min-ACC $H(y)$1, and avg-SG $H(y)$2, compared with MSGD at $H(y)$3, $H(y)$4, $H(y)$5, and Adam at $H(y)$6, $H(y)$7, $H(y)$8. On Rotated MNIST, NGM reached $H(y)$9, $g_i(t)$0, $g_i(t)$1, compared with MSGD at $g_i(t)$2, $g_i(t)$3, $g_i(t)$4, and Adam at $g_i(t)$5, $g_i(t)$6, $g_i(t)$7. On Split MNIST, NGM recorded $g_i(t)$8, $g_i(t)$9, $W_{ij}(t)=g_i(t)\,w_{ij}(t)$0, compared with MSGD at $W_{ij}(t)=g_i(t)\,w_{ij}(t)$1, $W_{ij}(t)=g_i(t)\,w_{ij}(t)$2, $W_{ij}(t)=g_i(t)\,w_{ij}(t)$3, and Adam at $W_{ij}(t)=g_i(t)\,w_{ij}(t)$4, $W_{ij}(t)=g_i(t)\,w_{ij}(t)$5, $W_{ij}(t)=g_i(t)\,w_{ij}(t)$6. On Split CIFAR-10, NGM achieved $W_{ij}(t)=g_i(t)\,w_{ij}(t)$7, $W_{ij}(t)=g_i(t)\,w_{ij}(t)$8, $W_{ij}(t)=g_i(t)\,w_{ij}(t)$9, compared with MSGD at $H(y)$00, $H(y)$01, $H(y)$02, and Adam at $H(y)$03, $H(y)$04, $H(y)$05. On Domain CIFAR-100, NGM produced $H(y)$06, $H(y)$07, $H(y)$08, compared with MSGD at $H(y)$09, $H(y)$10, $H(y)$11, and Adam at $H(y)$12, $H(y)$13, $H(y)$14.

The mechanistic interpretation given in the study has three parts: fast–slow dynamics, energy-landscape flattening, and uncertainty-guided adaptation. Gain spikes occur sharply at task switches, are higher for more complex shifts, and then decay to a new baseline that reflects accumulated task complexity. Test-loss peaks at transitions are lower, consistent with a flattened loss landscape. The stated limitations are equally specific: the benchmarks are limited to MNIST/CIFAR variants, only $H(y)$15–$H(y)$16 tasks were tested, and on CIFAR the gain was applied only to the classification head to avoid disrupting backbone feature learning.

5. Entropy-modulated policy gradients for long-horizon agents

A related but distinct formulation appears in long-horizon LLM agents trained from sparse outcome rewards. The starting point is the claim that the magnitude of policy gradients is inherently coupled with the entropy, producing inefficient small updates for confident correct actions and potentially destabilizing large updates for uncertain ones. EMPG addresses this by re-calibrating each step’s learning signal according to step-wise uncertainty and final task outcome (Wang et al., 11 Sep 2025).

The behavioral logic of the method is explicit. For confident and correct steps, $H(y)$17 and $H(y)$18 is low, so $H(y)$19 and the positive gradient is amplified. For confident but wrong steps, $H(y)$20 and $H(y)$21 is low, so the negative correction is strengthened, which is intended to combat “hallucinated confidence.” For uncertain steps, $H(y)$22 is high, so $H(y)$23 and noisy exploratory updates are attenuated. The future-clarity bonus

$H(y)$24

adds an intrinsic reward to transitions that lead to more predictable next steps.

The empirical evaluation used WebShop, ALFWorld, and Deep Search, with GRPO and DAPO as baselines. The key metrics were success rate on ALFWorld subtasks and overall, average score and success rate on WebShop, and retrieval/synthesis accuracy on Deep Search for in-domain, out-of-domain, and overall splits. The main gains reported were: on ALFWorld with a $H(y)$25B model, GRPO $H(y)$26 pp and DAPO $H(y)$27 pp; on WebShop, DAPO $H(y)$28B success $H(y)$29 pp and $H(y)$30B success $H(y)$31 $H(y)$32 pp$H(y)$33; on Deep Search, DAPO $H(y)$34B overall $H(y)$35 $H(y)$36 pp$H(y)$37, with ID $H(y)$38 pp and OOD $H(y)$39 pp.

The additional benefits reported are a substantial reduction in late-stage KL spikes, synergy between gradient-scaling and the clarity bonus, and better generalization on OOD data by learning “how to handle uncertainty.” A plausible implication is that uncertainty-modulated gain need not be attached to a physical feedback loop; it can operate directly on the estimator of policy improvement itself.

6. Control-theoretic, Bayesian, and conceptual extensions

In robust control, Liang–Zhao–Qiu study feedback stability under simultaneous gain and phase constraints through the sectored-disk uncertainty set

$H(y)$40

The corresponding static matrix problem is formulated through the Davis–Wielandt shell

$H(y)$41

and feedback robust stability is reduced to checking whether $H(y)$42 intersects $H(y)$43. For the symmetric case, $H(y)$44 with $H(y)$45, $H(y)$46, and $H(y)$47. This yields sufficient LMI-style conditions with nonnegative multipliers $H(y)$48, and in the half-disk limit $H(y)$49 a necessary-and-sufficient test is recovered. In practice, the LMIs quantify how much gain dynamics the system can endure under worst-case phase uncertainty, and frequency-wise multipliers reveal the most critical frequencies (Liang et al., 2024).

A distinct Bayesian result shows that not all uncertainty is alike. In a Gaussian state-space bandit with latent state evolution

$H(y)$50

and observation model

$H(y)$51

Kalman filtering yields the time-varying gain

$H(y)$52

which plays exactly the role of a learning rate. At stationarity,

$H(y)$53

with $H(y)$54 and $H(y)$55. Higher volatility raises equilibrium gain, whereas higher stochasticity lowers it. The CAUSE index inherits the same monotonicities in its exploration bonus, so volatility enhances exploration while stochasticity suppresses it (Piray, 19 May 2026). This directly qualifies any oversimplified view that uncertainty should uniformly increase gain.

At a more abstract level, Dahlsten and collaborators prove a trade-off between certainty and non-classical dynamics in a two-path interferometer. With certainty parameter $H(y)$56, if $H(y)$57 then the set of branch-local, $H(y)$58-preserving transformations collapses to the identity, whereas if $H(y)$59 one recovers the full quantum phase group $H(y)$60. The resulting “gain” function is

$H(y)$61

This suggests a broad conceptual principle: uncertainty can be an enabling condition for dynamical freedom rather than merely a source of degradation (Dahlsten et al., 2012).

Taken together, these lines of work present uncertainty-modulated gain dynamics as a general design principle rather than a domain-specific trick. In motor control it separates reactive stabilization from predictive retuning; in continual learning it approximates fast and slow plasticity channels; in long-horizon reinforcement learning it reweights policy updates by confidence; in Bayesian filtering it becomes the learning rate itself; and in robust control it is bounded through explicit geometric and LMI conditions. The principal caveat, stated differently across these literatures, is that the form and source of uncertainty matter: reactive gain, predictive gain, volatility-sensitive gain, stochasticity-sensitive gain, and entropy-based gain are not interchangeable.