Random-Reset Policy Optimization for Noise Synchronization

• The paper presents a phase-reduction framework showing how colored noise drives synchronization by reducing the dynamics to an effective white-noise model with modified diffusion coefficients.
• It demonstrates that the Fourier expansion of phase sensitivity functions and the noise power spectrum determine the stationary phase difference and clustering phenomena.
• Numerical examples with FitzHugh–Nagumo and Stuart–Landau oscillators verify that spectral matching of noise components induces synchronized and multi-cluster states.

The paper proposes a general phase-reduction framework for understanding synchronization of uncoupled limit-cycle oscillators driven by a common colored noise source with an arbitrary spectrum, together with independent colored noises on each oscillator. The central idea is that, under weak noise and short correlation times relative to the phase-diffusion time scale, the colored-noise-driven phase dynamics can still be treated by an effective white-noise Langevin / Fokker–Planck description, but with diffusion coefficients that explicitly retain the noise power spectrum. This leads to a stationary phase-difference distribution whose shape is controlled by the overlap between the oscillator’s phase sensitivity function and the noise spectrum at harmonics of the oscillator frequency.

The starting model is an ensemble of $N$ identical limit-cycle oscillators

$$\dot{\bm{X}}^{(j)} = \bm{F}(\bm{X}^{(j)}) + \sqrt{D}\,\bm{G}(\bm{X}^{(j)})\bm{\xi}(t) + \sqrt{\epsilon}\,\bm{H}(\bm{X}^{(j)})\bm{\eta}^{(j)}(t), \qquad j=1,\ldots,N,$$

where $\bm{F}$ has a stable $T$-periodic orbit, $\bm{\xi}(t)$ is the common colored noise, and $\bm{\eta}^{(j)}(t)$ are independent colored noises. The key assumptions are: the noises are zero-mean, mutually independent across common/individual channels and across different oscillators, and each can be represented as a filtered white noise process; also, their correlation times are short compared with the phase diffusion time scale. The noise statistics are encoded by correlation matrices

$$\bm{C}_{\xi}(\tau)=\langle \bm{\xi}(t)\bm{\xi}(t-\tau)^{\top}\rangle,\qquad \bm{C}_{\eta}(\tau)=\langle \bm{\eta}^{(j)}(t)\bm{\eta}^{(j)}(t-\tau)^{\top}\rangle.$$

Under weak noise $(D\ll 1,\ \epsilon\ll 1)$, phase reduction yields

$$\dot{\phi}^{(j)} = \omega + \sqrt{D}\,\bm{Z}_{\rm G}(\phi^{(j)})\cdot\bm{\xi}(t) +\sqrt{\epsilon}\,\bm{Z}_{\rm H}(\phi^{(j)})\cdot\bm{\eta}^{(j)}(t) + O(D,\epsilon),$$

with natural frequency $\omega=2\pi/T$ and phase sensitivity functions $$\dot{\bm{X}}^{(j)} = \bm{F}(\bm{X}^{(j)}) + \sqrt{D}\,\bm{G}(\bm{X}^{(j)})\bm{\xi}(t) + \sqrt{\epsilon}\,\bm{H}(\bm{X}^{(j)})\bm{\eta}^{(j)}(t), \qquad j=1,\ldots,N,$$0. The analysis then focuses on the phase difference $$\dot{\bm{X}}^{(j)} = \bm{F}(\bm{X}^{(j)}) + \sqrt{D}\,\bm{G}(\bm{X}^{(j)})\bm{\xi}(t) + \sqrt{\epsilon}\,\bm{H}(\bm{X}^{(j)})\bm{\eta}^{(j)}(t), \qquad j=1,\ldots,N,$$1. Using the effective white-noise description of Nakao et al., the PDF $$\dot{\bm{X}}^{(j)} = \bm{F}(\bm{X}^{(j)}) + \sqrt{D}\,\bm{G}(\bm{X}^{(j)})\bm{\xi}(t) + \sqrt{\epsilon}\,\bm{H}(\bm{X}^{(j)})\bm{\eta}^{(j)}(t), \qquad j=1,\ldots,N,$$2 obeys an effective Fokker–Planck equation

$$\dot{\bm{X}}^{(j)} = \bm{F}(\bm{X}^{(j)}) + \sqrt{D}\,\bm{G}(\bm{X}^{(j)})\bm{\xi}(t) + \sqrt{\epsilon}\,\bm{H}(\bm{X}^{(j)})\bm{\eta}^{(j)}(t), \qquad j=1,\ldots,N,$$3

Because the two oscillators are identical and the common noise is unbiased, the drift vanishes: $$\dot{\bm{X}}^{(j)} = \bm{F}(\bm{X}^{(j)}) + \sqrt{D}\,\bm{G}(\bm{X}^{(j)})\bm{\xi}(t) + \sqrt{\epsilon}\,\bm{H}(\bm{X}^{(j)})\bm{\eta}^{(j)}(t), \qquad j=1,\ldots,N,$$4 The phase-difference diffusion is determined from correlation integrals of the phase velocities. After expansion in $$\dot{\bm{X}}^{(j)} = \bm{F}(\bm{X}^{(j)}) + \sqrt{D}\,\bm{G}(\bm{X}^{(j)})\bm{\xi}(t) + \sqrt{\epsilon}\,\bm{H}(\bm{X}^{(j)})\bm{\eta}^{(j)}(t), \qquad j=1,\ldots,N,$$5 and $$\dot{\bm{X}}^{(j)} = \bm{F}(\bm{X}^{(j)}) + \sqrt{D}\,\bm{G}(\bm{X}^{(j)})\bm{\xi}(t) + \sqrt{\epsilon}\,\bm{H}(\bm{X}^{(j)})\bm{\eta}^{(j)}(t), \qquad j=1,\ldots,N,$$6, the leading-order result becomes

$$\dot{\bm{X}}^{(j)} = \bm{F}(\bm{X}^{(j)}) + \sqrt{D}\,\bm{G}(\bm{X}^{(j)})\bm{\xi}(t) + \sqrt{\epsilon}\,\bm{H}(\bm{X}^{(j)})\bm{\eta}^{(j)}(t), \qquad j=1,\ldots,N,$$7

where

$$\dot{\bm{X}}^{(j)} = \bm{F}(\bm{X}^{(j)}) + \sqrt{D}\,\bm{G}(\bm{X}^{(j)})\bm{\xi}(t) + \sqrt{\epsilon}\,\bm{H}(\bm{X}^{(j)})\bm{\eta}^{(j)}(t), \qquad j=1,\ldots,N,$$8

$$\dot{\bm{X}}^{(j)} = \bm{F}(\bm{X}^{(j)}) + \sqrt{D}\,\bm{G}(\bm{X}^{(j)})\bm{\xi}(t) + \sqrt{\epsilon}\,\bm{H}(\bm{X}^{(j)})\bm{\eta}^{(j)}(t), \qquad j=1,\ldots,N,$$9

These formulas are the paper’s general framework: they extend the white-noise synchronization theory to arbitrary colored noises, with the synchronization statistics depending explicitly on the temporal correlations of the drive.

The stationary phase-difference distribution $\bm{F}$0 follows by setting $\bm{F}$1. Since the drift is zero, the stationary solution is proportional to the inverse diffusion: $\bm{F}$2 with normalization constants $\bm{F}$3 and $\bm{F}$4. This is the key analytical expression: peaks in $\bm{F}$5 occur where $\bm{F}$6 is small, i.e. where the common-noise contribution $\bm{F}$7 is minimized. In practice this means stable phase-locking or clustering states are determined by the minima of the effective diffusion function.

To make the dependence on the noise spectrum explicit, the paper expands the phase sensitivity functions in Fourier series,

$\bm{F}$8

and defines the power-spectrum matrices

$\bm{F}$9

Then

$T$0

with coefficients

$T$1

This is the paper’s general spectral selection principle: only the noise power near the harmonics $T$2 of the oscillator frequency matters for synchronization. In other words, colored noise contributes most strongly when its spectrum has weight at frequencies that match the oscillator’s intrinsic harmonics.

The mechanism for synchronized and clustered states is therefore spectral. If the common colored noise has a strong peak near the natural frequency $T$3, then $T$4 becomes large in the corresponding harmonic and the effective diffusion $T$5 is minimized near $T$6, yielding a synchronized state. If the dominant noise component is near $T$7, higher-harmonic structure in $T$8 can produce $T$9-cluster states, i.e. multiple stable peaks in $\bm{\xi}(t)$0 separated by $\bm{\xi}(t)$1. The simulations in the paper show this explicitly: for the FitzHugh–Nagumo oscillator, noise peaked near $\bm{\xi}(t)$2 yields a 3-cluster distribution. More generally, the stationary PDF becomes multimodal whenever $\bm{\xi}(t)$3 develops multiple minima due to the harmonic content of $\bm{\xi}(t)$4.

A particularly important conclusion is that colored noise can induce clustering even when the perturbation is additive. In the white-noise limit, clustered states in the cited theory arise only through multiplicative noise; here, because colored noise has temporal structure, the phase diffusion depends on the spectral overlap with the phase response even for additive coupling, so additive colored noise can also create clustered states. This is one of the main ways the colored-noise mechanism differs from white-noise-induced synchronization.

For white noise, the correlation functions reduce to delta functions,

$\bm{\xi}(t)$5

and the formulas collapse to the known white-noise result of Nakao et al. In that case, the synchronization/clustering statistics depend only on instantaneous correlations, not on a spectral matching condition. By contrast, in the colored-noise theory the PDF depends on the full power spectrum through $\bm{\xi}(t)$6 and $\bm{\xi}(t)$7, which makes the synchronization strength frequency-selective. The paper’s examples show this clearly: for a Stuart–Landau oscillator, synchronization is strongest when the common noise peak frequency matches the oscillator frequency; for other peak frequencies the effective coupling weakens. Thus the general framework is not just a colored-noise extension of white-noise theory, but a frequency-resolved theory in which synchronization is controlled by spectral resonance between noise and oscillator phase response.