---
title: Power-Law Decaying Feedback Kernel
url: https://www.emergentmind.com/topics/power-law-decaying-feedback-kernel
type: topic
---

# Power-Law Decaying Feedback Kernel

Searching arXiv for the cited papers and closely related terminology.
A power-law decaying feedback kernel is a causal or history-dependent weighting function whose contribution from past states decays algebraically with lag rather than on a single exponential timescale. In the cited literature, this object appears in several mathematically distinct forms: as the memory kernel in generalized Langevin equations (GLEs), as a spike-triggered refractory or prediction kernel in predictive coding, as a memory kernel in reinforced stochastic processes, and as a forgetting or response kernel in learning dynamics [1708.04995] [1010.6178] [2105.10653] [2602.06797]. A separate but closely related usage concerns kernels whose **spectra** decay by a power law, where the decaying object is the eigenvalue sequence of a kernel integral operator rather than a temporal kernel value itself [2309.13337].

## 1. Conceptual scope and terminology

In its literal dynamical sense, a feedback kernel enters an evolution equation through a history integral or an equivalent temporal weighting rule. In the GLE derived from a coarse-grained lattice, the feedback term is
\[
\int_0^t \Theta(t-\tau)p(\tau)\,d\tau,
\]
so \(\Theta\) is the memory kernel that feeds past coarse-grained velocities back into the present dynamics [1708.04995]. In predictive spiking models, each spike inserts a delayed causal kernel into the internal reconstruction,
\[
\hat{x}(t)=\sum_{t_j<t}\kappa(t-(t_j+\Delta)),
\]
so the same kernel acts as a refractory response and as a predictive feedback signal [1010.6178]. In the generalized Pólya urn, the kernel \(d(t)\) weights the influence of past draws in the current draw probability, which is again a direct feedback mechanism [2105.10653]. In the functional scaling law for SGD, past noise contributes to final loss through
\[
\mathcal K(t-\tau)=(1+t-\tau)^{-(2-1/\beta)},
\]
which is the closest analogue of a temporal feedback or forgetting kernel in that setting [2602.06797].

A common source of ambiguity is that not every “power-law kernel” result concerns a literal temporal kernel value. In kernel ridge regression, the central assumption is instead
\[
c_\beta i^{-\beta}\le \lambda_i\le C_\beta i^{-\beta},
\]
where \(\lambda_i\) are the eigenvalues of the kernel integral operator. Here the relevant power law is spectral, not temporal, and the paper does **not** use “feedback kernel” in a technical sense [2309.13337]. Likewise, in random-feature regression dynamics, the power law is imposed on the mode variances \(\eta_i=\eta_1 i^{-1-\gamma}\), while the temporal memory kernel \(K\) is generated self-consistently from the dynamics rather than assumed to decay as a power law in time [2602.23039].

## 2. Generalized Langevin equations and coarse-grained lattice dynamics

A mathematically explicit instance of a power-law decaying feedback kernel is obtained by coarse-graining an infinite one-dimensional harmonic lattice with nearest- and second-nearest-neighbor interactions. After linearization,
\[
\ddot{u}=-\mathcal A u,
\]
with
\[
(\mathcal A u)_j = -\kappa_2 u_{j-2}-\kappa_1 u_{j-1}+\kappa_0 u_j-\kappa_1 u_{j+1}-\kappa_2 u_{j+2},
\]
where \(\kappa_1=\frac12\varphi''(a_0)\), \(\kappa_2=\frac12\varphi''(2a_0)\), and \(\kappa_0=2(\kappa_1+\kappa_2)\). Under the stability assumptions \(\kappa_1>0\) and \(\kappa_1+4\kappa_2>0\), the Mori–Zwanzig formalism yields the exact coarse-grained GLE
\[
\dot q=p,\qquad
\dot p=-\mathcal A q-\int_0^t \Theta(t-\tau)p(\tau)\,d\tau+R(t),
\]
with random force \(R(t)\) satisfying
\[
\langle R(t)R(t')^\intercal\rangle = k_B T\,\Theta(t-t')
\]
through the second fluctuation-dissipation theorem [1708.04995].

The kernel itself is matrix-valued:
\[
\Theta(t)=\Phi^\intercal \mathcal A \Psi \cos(\Omega t)\Omega^{-2}\Psi^\intercal \mathcal A\Phi,
\qquad
\Omega^2=\Psi^\intercal \mathcal A \Psi.
\]
Because the lattice and coarse-graining are translationally invariant, the entries \(\Theta_{k\ell}(t)\) admit a block-Fourier representation. For the diagonal temporal kernel, the central asymptotic result is that, for piecewise constant coarse-graining,
\[
|\Theta_{0,0}(t)|=\mathcal O(t^{-1/2})\qquad (t\to+\infty),
\]
which the authors describe as decay “with rate at least equal to \(0.5\)” [1708.04995]. The result is an upper-order asymptotic estimate obtained by stationary phase; the paper does **not** provide a full explicit constant \(C\) in an asymptotic formula \(\Theta_{0,0}(t)\sim Ct^{-1/2}\), nor a matching lower bound.

The same analysis also distinguishes temporal and spatial behavior sharply. For piecewise constant averaging,
\[
|\Theta_{0,J}(0)|\le o(|J|^{-n})\qquad\text{for any positive integer }n,
\]
so spatial decay is faster than any algebraic power, while the temporal tail is slow [1708.04995]. Numerically, the paper reports exponential-looking spatial decay, which accentuates the contrast between fast spatial localization and persistent temporal memory. At \(t=0\), the kernel magnitude depends on the coarse-graining level \(M\): with piecewise constant averaging,
\[
0\le \Theta_{0,0}(0)\le \frac{2\kappa_1+4\kappa_2}{M},
\]
hence \(\Theta_{0,0}(0)=\mathcal O(M^{-1})\), whereas for piecewise linear averaging the estimate is \(\Theta_{0,0}(0)=\mathcal O(M^{-2})\) [1708.04995]. The rigorously established \(t^{-1/2}\) tail, however, is only proved for the piecewise constant case.

## 3. Power-law memory and phase transition in reinforced stochastic processes

In the generalized Pólya urn with memory kernel, the feedback variable is a weighted empirical average of past binary states \(X(t)\in\{0,1\}\). With
\[
D_d(t)=\sum_{s=1}^t d(s-1),\qquad
S_d(t)=\sum_{s=1}^t X(s)\,d(t-s),\qquad
Z_d(t)=\frac{S_d(t)}{D_d(t)},
\]
the draw probability becomes
\[
P(X(t+1)=1\mid X(1),\dots,X(t))
=\frac{\alpha+S_d(t)}{\theta+D_d(t)}
=\frac{\alpha+D_d(t)Z_d(t)}{\theta+D_d(t)}.
\]
The kernel \(d(\tau)\) therefore controls how strongly lag-\(\tau\) observations feed back into present reinforcement [2105.10653].

For the power-law case,
\[
d(t)=(1+t)^{-\gamma},
\]
the asymptotics of the cumulative memory weight are
\[
D_\gamma(t)\sim
\begin{cases}
\dfrac{1}{\gamma-1}, & \gamma>1,\\[1ex]
\ln t, & \gamma=1,\\[1ex]
\dfrac{t^{1-\gamma}}{1-\gamma}, & \gamma<1.
\end{cases}
\]
This leads to a phase transition at \(\gamma_c=1\). The autocorrelation function
\[
C(t)=\frac{\sum_{s=1}^{t}C(s-1)d(t-s)}{\theta+D_d(t)}
\]
has order parameter \(c=\lim_{t\to\infty}C(t)\), and the paper reports \(c>0\) for \(\gamma<1\), \(c=0\) for \(\gamma>1\), and critical algebraic decay at \(\gamma=1\) [2105.10653]. The long-time exponents are summarized as
\[
(\delta,\delta')=
\begin{cases}
(\gamma,\,2\gamma-1), & \gamma>1,\\[1ex]
(\delta_c,\,\delta_c), & \gamma=1,\\[1ex]
(0,\,1-\gamma), & \gamma<1.
\end{cases}
\]
Thus the power-law decay exponent changes discontinuously at the critical point.

The paper also contrasts this behavior with exponentially decaying memory. For \(d(t)=e^{-rt}\), the total memory weight converges, the effective state variable obeys an Ornstein–Uhlenbeck equation, and the autocorrelation decays exponentially:
\[
C(t)\sim e^{-t/\xi}.
\]
By contrast, the power-law kernel is represented as a continuum mixture of exponentials,
\[
(t+1)^{-\gamma} = \int_0^\infty e^{-rt} f_\gamma(r)\,dr,
\qquad
f_\gamma(r)=\frac{r^{\gamma-1}e^{-r}}{\Gamma(\gamma)},
\]
which produces a Markovian embedding in an infinite-dimensional auxiliary field \(\{Z(r,t)\}\) driven by a common Wiener process [2105.10653]. This suggests that the nontrivial long-memory behavior is not merely a consequence of slow decay, but of the collective effect of infinitely many slow modes coupled through shared noise.

## 4. Predictive spiking neurons, refractory kernels, and fractional derivatives

In predictive spiking models, the power-law decaying feedback kernel is a spike-triggered refractory or prediction kernel. The idealized form is a shifted power law,
\[
(t-t_i)^{-\beta},
\]
and the practical regularized version is
\[
\kappa(t)=\lambda v(t,k)t^{-\beta},
\qquad
v(t,k)=2\,\text{logsig}(kt)-1,
\qquad
\text{logsig}(t)=\frac{1}{1+\exp(-t)}.
\]
The kernel is causal, zero before onset, and characterized by exponent \(\beta\), onset parameter \(k\), and scale \(\lambda\) [1010.6178]. With positive and negative spikes, the reconstruction is
\[
\hat{x}(t)
=
\sum_{t_j^+<t}\kappa(t-(t_j^+ +\Delta))
-\sum_{t_j^-<t}\kappa(t-(t_j^- +\Delta)).
\]
Each emitted spike therefore adds a long-tailed kernel to the internally reconstructed signal and suppresses future spikes over many timescales.

The same model admits a fractional-calculus interpretation. If
\[
r(t)=D^\alpha x(t),
\]
then the paper argues that when a signal is approximated by a sum of power-law kernels with exponent \(\beta\), the spike train behaves as the fractional derivative of order
\[
\alpha=1-\beta.
\]
The key Fourier-domain statement is that the transform of \(t^{-\beta}\) is proportional to \((i\omega)^{\beta-1}\); thus applying fractional differentiation of order \(\alpha=1-\beta\) maps the kernel to a constant spectrum, corresponding to a Dirac spike [1010.6178]. The paper explicitly notes that “a spike-train is the \(\alpha=0.2\) fractional derivative of a signal approximated by a sum of power-law kernels with exponent \(\beta=0.8\).”

The power-law kernel is contrasted directly with exponentially decaying kernels in online coding. For synthesized fractional Brownian motion signals with \(H>0.5\), the paper reports that power-law kernels required **less than half the number of spikes** for similar SNR compared to similar exponentially decaying kernels; one figure caption gives **1398 spikes** for exponential-kernel encoding versus **618 spikes** for power-law-kernel encoding at the same SNR [1010.6178]. The paper adds an important caveat: without negative spikes, the advantage can disappear because a slow power-law tail cannot retract quickly on descending signals. For implementation, the paper states that the normalized power-law kernel can be approximated very accurately over multiple orders of magnitude by a sum of just **11 \(\alpha\)-function exponentials**, and that this decomposition supports downstream temporal filtering by selectively suppressing slow or fast exponential components [1010.6178].

## 5. Spectral power laws, collective response kernels, and optimization dynamics

In learning theory, “power-law decaying kernel” frequently refers to spectral decay. For kernel ridge regression, the kernel integral operator
\[
(Tf)(x)=\int_X k(x,y)f(y)\,d\mu(y)
\]
has eigenvalues satisfying
\[
c_\beta i^{-\beta}\le \lambda_i\le C_\beta i^{-\beta},
\qquad \beta>1.
\]
Under this assumption and a source condition
\[
f_\rho^*=\sum_{i=1}^\infty a_i \lambda_i^{s/2} i^{-1/2} e_i,
\qquad 0<c\le |a_i|\le C,
\]
the excess risk follows the asymptotic form
\[
R(\lambda)\approx \lambda^{\min(s,2)}+\frac{\sigma^2}{n}\lambda^{-1/\beta}
\quad\text{for }\lambda=\Omega(n^{-\beta}),
\]
while in the nearly interpolating regime \(\lambda=O(n^{-\beta})\) one has
\[
R(\lambda)\gtrsim \sigma^2
\]
in the noisy case [2309.13337]. The effective dimension satisfies \(\mathcal N(\lambda)\asymp \lambda^{-1/\beta}\), so the power law controls variance through the number of active spectral directions.

A related but dynamically richer construction appears in random-feature regression with power-law-distributed kernel eigenvalues
\[
\eta_i=\eta_1\, i^{-1-\gamma}.
\]
After disorder averaging and dynamical mean-field reduction, the mode discrepancies \(v_i(t)=\bar w_i-w_i(t)\) satisfy
\[
\left[\partial_t+\frac{1}{g\beta}\right]v_i(t)+P\eta_i\int K(t-s)\,v_i(s)\,ds
=
\frac{\bar w_i}{g\beta}+\xi_i(t),
\]
where \(K\) is a self-consistent collective memory kernel defined by
\[
K=\bar R*K+\delta,
\qquad
\hat K(\omega)=\frac{1}{1-\hat R(\omega)}.
\]
The paper explicitly interprets this as a nonlocal self-coupling and as an Onsager reaction term [2602.23039]. It does **not** claim that \(K(t)\) itself decays as a power law in time; rather, the power-law spectrum \(\eta_i\sim i^{-1-\gamma}\) generates a broad hierarchy of mode time scales that shapes the time dependence of the feedback kernel and the resulting bias–variance dynamics.

The functional scaling law for one-pass SGD makes the temporal-memory interpretation fully explicit. Under
\[
\lambda_j \eqsim j^{-\beta},
\qquad
\lambda_j |\theta_j^*|^2 \eqsim j^{-(1+s\beta)},
\]
the expected excess risk is modeled as
\[
\mathbb E[\mathcal E(\bar\theta_t)]
\eqsim
(1+t)^{-s}
+
\int_0^t \mathcal K(t-\tau)\,\varphi(\tau)\,d\tau,
\qquad
\mathcal K(t)=(1+t)^{-(2-1/\beta)}.
\]
Here \(\mathcal K\) is a power-law forgetting kernel for injected noise [2602.06797]. Solving the finite-horizon schedule optimization yields the easy-task optimal learning-rate schedule
\[
\eta^*(z)=\eta_{\mathrm{peak}}\left(1-\frac{z}{N}\right)^{2\beta-1}
\]
up to the stated asymptotic correction, and a warmup-stable-decay structure in the hard-task regime \(s<1-1/\beta\) [2602.06797]. A common misconception is therefore to identify every power-law kernel in learning theory with a temporal convolution kernel; in some cases the power law is spectral, in some it is a forgetting kernel, and in some it shapes a self-consistent response operator.

## 6. Approximation theory, perturbation bounds, and limitations

For stochastic Volterra formulations of GLEs, power-law decaying feedback kernels are treated through weighted Banach spaces and weighted Schur norms. The first-order model
\[
dV_t = -\gamma V_t\,dt -\int_0^t K(t,s)V_s\,ds\,dt + \sigma dB_t
\]
and its perturbed analogue with kernel \(\tilde K\) are compared under synchronized noise coupling \(B_t=\tilde B_t\). The central kernel norm is
\[
|K|_h := \sup_{t\ge 0}
\left(
\int_0^t \frac{\|K(t,s)\|^2}{h(t-s)}\,ds
\right)^{1/2},
\]
with \(h\in U(\mu)\), \(\mu\le 0\), and the paper states explicitly that examples of \(U(0)\) include
\[
h(t)\sim (1+t)^{-\alpha},\qquad \alpha>1.
\]
Thus integrable power-law tails are directly admissible [2512.10256].

For translation-invariant power-law kernels
\[
K(t)\sim c(1+t)^{-\alpha},
\]
the framework applies naturally when \(\alpha>1\). If \(h(t)\asymp (1+t)^{-\beta}\) with \(1<\beta<2\alpha-1\), then the weighted Schur norms are finite, and the trajectory discrepancy satisfies a bound of the form
\[
\mathbb E|V_t-\tilde V_t|^2
\lesssim
\bigl(\mathbb E|V_0-\tilde V_0|^2+|\delta K|_h^2\bigr)h(t)
+
|\delta K|_h^2\operatorname{Tr}(\sigma\sigma^\top),
\]
with analogous estimates for the second-order GLE in a hypocoercive Lyapunov-type distance [2512.10256]. The paper’s explicit scalar example
\[
k(t)=c(t+\alpha)^{-\beta}
\]
shows that if \(a>\int_0^\infty k(t)\,dt\), then the Volterra comparison theorem yields \(y(t)\lesssim t^{-\beta}\) when \(g\equiv 0\). This means that, in the integrable power-law regime, the decay rate of the perturbation bound is directly controlled by the chosen kernel envelope.

The numerical evidence in that work uses
\[
K(t)=(1+t)^{-4},\qquad h(t)=(1+t)^{-6},
\]
and reports that the observed trajectory-error decay is close to exponent \(8\), near the decay of \(K^2\), even though the proven envelope uses exponent \(6\) [2512.10256]. This suggests that the weighted-space theory can be conservative. The same paper also sets a clear limit on its scope: the subexponential framework does **not** directly cover nonintegrable power laws with \(\alpha\le 1\) [2512.10256]. Comparable scope limitations appear elsewhere: the lattice GLE proof of \(t^{-1/2}\) is confined to a one-dimensional harmonic chain with piecewise constant coarse-graining [1708.04995]; the KRR learning-curve results concern spectral power laws rather than literal temporal feedback kernels [2309.13337]; and the random-feature DMFT derives a temporal response kernel from a power-law mode spectrum without proving that the response kernel itself has a power-law tail in time [2602.23039]. Taken together, these results support a narrow but technically clear conclusion: power-law decay in a feedback kernel can produce persistent non-Markovian memory, phase transitions, and polynomial stability or learning laws, but the precise object that decays by a power law must be identified case by case.

Source: https://www.emergentmind.com/topics/power-law-decaying-feedback-kernel