Other contributors to the upward bias in online review discussions

Determine whether user groups beyond accepted-paper authors (survivors), high-scoring rejected-paper authors (complainers), and borderline-score authors (borderliners) contribute to the upward bias observed in self-reported academic conference review score distributions on online discussion platforms such as Zhihu and Reddit, and ascertain their impact on the sample-versus-population discrepancy.

Background

The paper documents a consistent upward bias: review scores self-reported in online discussions (Zhihu and Reddit) are statistically higher than the population distributions for ICLR and ARR. The authors attribute this bias to three mechanisms: survivors (accepted authors sharing good news), complainers (high-scoring rejected authors voicing dissatisfaction), and borderliners (authors near the acceptance threshold seeking advice pre-decision).

Despite these identified mechanisms, the authors note uncertainty about whether additional types of users also contribute to the upward skew. Clarifying whether other contributors exist would refine the understanding of bias sources and improve bias correction methods and interpretation of online review threads.