Shaping the learning signal in a combined Q-learning rule to improve structured cooperation

Abstract: Q-learning provides a standard reinforcement learning framework for studying cooperation by specifying how agents update action values from repeated local interactions outcomes. Although previous work has shown that reputation can promote cooperation in such systems, most models introduce reputation by modifying payoffs, encoding it directly in the state or changing partner selection, which makes it difficult to isolate the role of the learning signal itself. Here, we construct the reinforcement signal as a weighted combination of reputation and game payoffs, leaving the game and network structure unchanged. We find that increasing the weight on reputation generally promotes cooperation by consolidating clusters, but this effect is conditional on the learning dynamics. Specifically, this promoting effect vanishes in two regimes: when the learning rate is extremely small, which prevents effective information propagation and when the discount factor approaches one, as distant future expectations obscure the immediate reputational advantage. Outside these limiting cases, the efficacy of reputation in promoting cooperation is attenuated by higher learning rates but amplified by larger discount factors. These results advance the understanding of cooperative dynamics by demonstrating that cooperation can be stabilized through the reputational shaping of learning signals alone, providing critical insights into the interplay between social information and individual learning parameters.