Double blind vs. open review: an evolutionary game logit-simulating the behavior of authors and reviewers

Abstract: Despite the tremendous successes of science in providing knowledge and technologies, the Replication Crisis has highlighted that scientific institutions have much room for improvement. Peer-review is one target of criticism and suggested reforms. However, despite numerous controversies peer review systems, plus the obvious complexity of the incentives affecting the decisions of authors and reviewers, there is very little systematic and strategic analysis of peer-review systems. In this paper, we begin to address this feature of the peer-review literature by applying the tools of game theory. We use simulations to develop an evolutionary model based around a game played by authors and reviewers, before exploring some of its tendencies. In particular, we examine the relative impact of double-blind peer-review and open review on incentivising reviewer effort under a variety of parameters. We also compare (a) the impact of one review system versus another with (b) other alterations, such as higher costs of reviewing. We find that is no reliable difference between peer-review systems in our model. Furthermore, under some conditions, higher payoffs for good reviewing can lead to less (rather than more) author effort under open review. Finally, compared to the other parameters that we vary, it is the exogenous utility of author effort that makes an important and reliable difference in our model, which raises the possibility that peer-review might not be an important target for institutional reforms.