Once-Reinforced Random Walk

• ORRW is a stochastic process on graphs where each edge’s weight increases only on its first traversal, affecting future transition probabilities.
• It exhibits a phase transition: low reinforcement leads to random-like exploration, while high reinforcement fosters localization and self-attraction.
• Analysis leverages coupling arguments, martingale techniques, and central limit theorems, with applications in network navigation and reinforcement learning.

A once-reinforced random walk (ORRW) is a stochastic process on a graph (typically $\mathbb{Z}^d$ or a more general network) in which the probability of traversing an edge depends on the local history of the walker, specifically the number of times each edge has been previously traversed. In the canonical ORRW, each edge of the underlying graph has an associated weight, which is incremented the first time the edge is traversed by the walker, but not on subsequent visits—hence the term “once-reinforced.” This local reinforcement induces a path-dependent dynamic that interpolates between simple random walk (no reinforcement) and more aggressively reinforced walks (such as the edge- or vertex-reinforced random walks with unbounded reinforcement).

1. Mathematical Definition

Let $G=(V,E)$ denote a locally finite, undirected graph. The process starts by placing weights $w_0(e)=1$ on each edge $e\in E$. At each time $n=0,1,2,\ldots$, the walker occupies a vertex $X_n\in V$. Conditional on the current position $X_n=v$, the probability of traversing edge $e=(v,u)$ is proportional to its current weight $w_n(e)$: $$\mathbb{P}(X_{n+1}=u\mid X_n=v; \{w_n(e)\}) = \frac{w_n(e)}{\sum_{e'=(v,\cdot)} w_n(e')}.$$ The weight update is local and “once-only”:

$G=(V,E)$0

where $G=(V,E)$1 is the reinforcement parameter.

2. Historical Context and Related Models

The once-reinforced random walk was introduced as an analytically tractable yet nontrivial variant of other reinforced processes, especially the edge-reinforced random walk (ERRW), in which the reinforcement grows with the number of traversals, not just the first. The ORRW thus occupies a critical middle ground between the ordinary random walk (no memory) and strongly reinforced models that can exhibit localization or trapping.

For general reinforcement schemes, the probability to choose an edge at time $G=(V,E)$2 depends on a function $G=(V,E)$3 of the local count, e.g., $G=(V,E)$4. In ORRW, $G=(V,E)$5.

3. Phase Behavior and Recurrence/Transience

A key question for the ORRW on infinite graphs such as $G=(V,E)$6 is whether the process is recurrent (returns to the origin infinitely often) or transient (escapes to infinity). On trees and $G=(V,E)$7, the walk remains recurrent for all $G=(V,E)$8. On $G=(V,E)$9 ($w_0(e)=1$0), the underlying random walk is transient; the effect of once-reinforcement is more subtle.

Analytic results indicate that, on regular trees, the ORRW exhibits a phase transition in the reinforcement parameter $w_0(e)=1$1. For small $w_0(e)=1$2, the walk is transient, with behavior asymptotically similar to the simple random walk. For large $w_0(e)=1$3, the reinforcement produces a self-attracting behavior: the walker is more likely to revisit previously traversed edges, leading to localization around certain clusters or traps. On $w_0(e)=1$4, however, the walk remains transient for all finite $w_0(e)=1$5.

4. Connections to Other Stochastic Processes

The ORRW shares connections with several stochastic models:

• Edge-Reinforced Random Walk: The classical ERRW allows the weights to keep increasing; the ORRW freezes reinforcement after one visit.
• Vertex-Reinforced Jump Process: Here, reinforcement is applied to vertices rather than edges.
• Excited Random Walks: Each site imparts a self-interaction to the walker based on the number of visits, but the effect may decay after some threshold.

These processes are unified by their local, history-dependent update rules, but diverge in their long-term behavior due to the reinforcement schedule.

5. Key Analytical and Probabilistic Tools

Analysis of the ORRW typically employs coupling arguments, martingale techniques, and sometimes comparison with electrical network models. The transition probabilities, modified by reinforcement, require nontrivial adaptations of standard random walk tools.

Results of interest include central limit theorems for the walker's position, large deviations properties, and laws of large numbers for various statistics. In some settings, the ORRW can be mapped to polymer models with nonlocal self-interaction, or to urn processes.

6. Applications and Implications

The primary importance of the ORRW lies in its role as a minimal model of path-dependent self-interacting randomness. Applications are primarily found in theoretical probability and statistical mechanics. In network science, reinforced processes—including the ORRW—offer paradigms for modeling exploration with memory, with possible implications for network navigation, optimal foraging, and reinforcement learning heuristics where exploration costs change after first traversal.

A plausible implication is that, due to the non-monotonic reinforcement, the ORRW provides insight into systems where novelty-seeking is rewarded only once, matching certain resource exploration or search scenarios.

7. Open Problems and Future Directions

Despite the comparative tractability of the ORRW versus other strongly reinforced models, many questions remain open, particularly for higher-dimensional graphs, random environments, and models with varying $w_0(e)=1$6 or reinforcement rules (e.g., edge- and vertex-hybrid reinforcement). Generalizations include multi-excited or multi-level reinforcement, and extensions to directed graphs or networks with edge inhomogeneity.

Further research aims to classify phase transitions in more complex topologies and to connect the ORRW to broader families of interacting particle systems and random environments.