Comparative effectiveness of WDYM gameplay and discussion

Determine how much participation in the What Do You Mean? (WDYM) game raises awareness of commit-message communication problems compared with discussing those problems without the game.

Background

The study evaluates the WDYM game as an educational intervention for increasing awareness of communication and quality problems in software-engineering commit messages. The research design does not include a control group receiving only a discussion-based treatment, so it cannot isolate the effect of gamification or determine whether the game is more effective than simply talking about the problem.

The authors explicitly identify this comparison as unresolved. Establishing the relative contribution of the WDYM game, as opposed to discussion alone, would clarify whether the game mechanics provide educational benefits beyond those produced by exposure to the topic itself.

References

Considering that we did not want to measure improvement, the addition of a control group could have given us another viewpoint. How much the wdym game helped raise awareness compared to talking about the problem is up for debate.

What Does It Mean and Why Should I Bother? Motivating Students to Write Better Commit Messages  (2608.17993 - Balogh et al., 18 Aug 2026) in Section 6, Threats to Validity