State-Dependent Random Walks

• State-dependent random walks are stochastic processes whose transition probabilities vary with the current state, environment, or past trajectory, enabling modeling of dynamic feedback and complex systems.
• These models include site-based feedback, dynamic environments, and nonlinear dynamics, which result in sharp phase transitions such as ballisticity and mixed recurrence regimes.
• Analytical techniques like martingale potential methods, renewal theory, and Lyapunov exponent computations provide insights into drift, invariant measures, and chaotic behavior in these processes.

State-dependent random walks are stochastic processes whose evolution incorporates—explicitly or through coupling—dependence on the current state, local environment, or trajectory history, yielding transition probabilities or update dynamics that deviate sharply from classical homogeneous or stationary models. This intrinsic state-dependence underpins an array of phenomena inaccessible to traditional Markovian analysis, including dynamic spatial feedback, environment adaptation, transport in varying media, and emergence of non-linear or even chaotic macroscopic behavior.

1. Mathematical Formulations and Model Taxonomy

A state-dependent random walk is any process $(X_n)_{n\geq0}$ on a (typically countable) state space $S$ such that the law of $X_{n+1}$ given $\mathcal{F}_n$ depends on the current or past values of $X_k$, on environmental or auxiliary processes $(\omega_k)$, or both. Representative frameworks include:

• Site-based Feedback Models: The transition kernel at each site is determined by a site-local Markov process $(\lambda(x))_{x\in\mathbb{Z}}$, possibly with counters and mode-switching, as in site-based reinforcement (Pinsky et al., 2014).
• Random Walk in Changing Environments: The underlying graph or edge-weight structure $G_t$ evolves jointly with the walk, and $X_{t+1}$ is drawn according to the walk's current position and $G_t$, yielding highly general forms encompassing reinforced random walks, bridge-burning walks, and self-avoiding walks (Amir et al., 2015).
• Random Walks in Dynamic/Random Environments: The environment itself (sites, particles, graphs) is a dynamic stochastic process (e.g., moving particles or Ornstein–Uhlenbeck velocities), and the walk's law depends on the instantaneous local environment (Hilário et al., 2014, Blondel et al., 2017, Michelot et al., 2018).
• Nonlinear Random Walks: Transition probabilities are nonlinear functions of the current occupation distribution, supporting phenomena such as chaos (Chitrakar et al., 2021).
• Deterministically Driven Walks: Evolutions are deterministic given a realization of a stationary ergodic environment, with recurrence/transience determined by typical ergodic averages (Little, 2013).
• State-Dependent Random Walks on Structured Spaces: For example, block-structured Markov chains or random walks on strips, wherein the horizontal transition structure depends on a vertical "level" process (Hong et al., 2013).

2. Fundamental Phenomena: Recurrence, Transience, Speed

In sharp contrast to standard random walks, state-dependent models exhibit phase transitions and sharp cutoffs depending not only on local drift but on the interplay between walk-local and environment-local evolution:

• Sharp Cutoff for Transience/Recurrence: In site-based feedback random walks, there exists a computable parameter $\alpha=\alpha(p, q, R, L)$, representing a “long-run rightward tendency.” The walk is almost surely transient to $+\infty$ if $\alpha>1/2$, to $-\infty$ if $\alpha<1/2$, with the recurrent or mixed transient regime at the critical point $\alpha=1/2$, further depending on the initial environment (Pinsky et al., 2014).
• Ballisticity and Speed: For $\alpha\neq 1/2$, positive speed (ballistic transport) emerges, with explicit formulas available in specialized regimes. For walks in dynamic random environments, speed can depend intricately on particle density, local permeability, and the nature (lazy/non-lazy) of the environment’s evolution (Hilário et al., 2014, Blondel et al., 2017).
• Recurrence/Transience under Changing Environments: On $\mathbb{N}$ or trees with monotone (increasing or decreasing) adaptive environments, recurrence and transience can be established with martingale potential arguments, employing harmonic functions tailored to the time-varying conductances (Amir et al., 2015).

3. State-Dependent Models in Dynamic and Structured Environments

State-dependence can arise from interaction with a dynamic or structured external process:

• Random Walk on Random Walks: The walker's local transition kernel depends on the occupancy of the current site by a system of independent random walkers (environment particles). This coupling generates regimes of ballisticity, the sign and magnitude of which shift discontinuously as particle density and the permeability of the environment are tuned (Blondel et al., 2017, Hilário et al., 2014).
• Continuous-Time State-Switching Models: In continuous-time correlated random walks, velocity (modelled as an Ornstein–Uhlenbeck process) and behavioral state (evolving as a CTMC) interact, with the transition kernel for $(X_t, V_t)$ depending on the latent state $s_t$. The inference procedure exploits the resulting state-space structure and yields interpretable behavioral parameters (velocity autocorrelation decay, mean speed, dwell times) (Michelot et al., 2018).
• Coupled Nonlinear Dynamics: When transition probabilities are general nonlinear functions of the system state, period-doubling bifurcations and robust chaos appear even on small graphs, with broad implications for transport and predictability in networked systems (Chitrakar et al., 2021).

4. Explicit Results: Drift, Invariant Measures, and Light-Tailed Distributions

• Drift Computation in Dependent Environments: For one-dimensional random walks in $k$-dependent or moving average environments, the asymptotic drift $V$ can be written in terms of the stationary law $\pi$ of an associated finite-state Markov chain $Y_n$ and the matrix $D$ of local biases:

$V=\frac{1}{2\,\pi (I-PD)^{-1}\mathbf1 - 1}$

where $P$ is the transition matrix of $Y_n$ and $D$ is diagonal in the state-based walk biases. Explicit quartic and higher-degree rational forms for $V$ are available in low-dimensional settings (Scheinhardt et al., 2014).

• Stationary Distribution for State-Dependent Random Walks on a Strip: For reflecting walks on a half-strip, Hong, Zhang, and Zhao provide explicit closed-form stationary measures $\pi_n(i)$ using recursively defined matrices $A_n$, $u_n$, with the light-tailed, geometric decay rate given by the Perron root $\lambda_{\max}(A)$ of the limiting up-matrix (Hong et al., 2013). Branching process decompositions reveal the underlying mechanisms with greater transparency than traditional matrix-analytic methods.

5. Hysteresis, Memory, and Non-Markovian Effects

• Higher-Order Dependence and Hysteresis: State dependence often manifests as memory or hysteresis, studied using higher-order Markov chains. Statistical selection of lag order (memory length) is subtle: Bayesian methods with leave-one-out cross-validation are recommended over Bayes factors with flat priors (which are biased in the large-data limit) and AIC (which can underfit when data are scarce) (Chang, 2017).
• Path-Dependent Models and Adaptive Local Bias: Random walks may modify their local environments (e.g., reinforced random walks, bridge burning), causing feedback loops with nontrivial long-term consequences: e.g., monotone adaptive walks on $\mathbb{Z}^2$ can be transient despite being uniformly bounded and proper, defying expectations from fixed-environment models (Amir et al., 2015).

6. Nonlinear and Chaotic State Dependence

• Chaos via Non-Monotonic State Dependence: If transition rates are non-monotonic (e.g., peaked at intermediate occupancies), the system’s simplex dynamics generically develop period-doubling cascades, chaos (positive nontrivial Lyapunov exponents), and strange attractors, even on regular or random networks. The observed loss of predictability and highly heterogeneous statistics of return times highlight the potential impact on information diffusion, resource allocation, and transport processes (Chitrakar et al., 2021).

7. Open Problems, Critical Regimes, and Unresolved Questions

• Critical and Intermediate Behavior: At critical points (e.g., $\alpha=1/2$ in site-based walks), the limit behavior can depend intricately on initial environment “boundary conditions,” with coexistence of recurrence and transience, and even mixtures of both for favored vs. disfavored sides (Pinsky et al., 2014).
• Non-trivial Divergence in State-Time-Dependent Walks: The construction of time-inhomogeneous, state-dependent random walks can yield paths that are neither recurrent nor almost-surely divergent except for extremal parameter regimes. For example, a stair-embedded walk driven by a deterministic sequence $\{a_n\}$ exhibits almost-sure divergence for constant, bounded $a_n$, yet admits only divergence probabilities arbitrarily close to, but strictly less than, one for sequences with $a_n\to\infty$. Full characterizations of such transition phenomena remain open (Li et al., 2018).
• Conjectures in Adaptive Environments: It is conjectured that in nonadaptive, monotone, bounded environments on recurrent base graphs, state-dependent walks cannot be made transient by time-varying but fixed-in-advance parameters. However, adaptivity—where the environment depends on the trajectory—can break this dichotomy (Amir et al., 2015).

---

State-dependent random walks constitute a fundamental generalization bridging classical probability, stochastic processes on dynamic structures, and non-equilibrium statistical physics. Their study reveals an array of non-classical behaviors, intricate phase transitions, and analytical challenges, with advanced tools drawn from renewal theory, branching processes, matrix-analytic techniques, Lyapunov exponent calculation, and Bayesian model comparison. Ongoing research targets new universality classes, comprehensive classifications of critical phenomena, and practical inference frameworks for complex state-dependent transport processes.