Determine the bias introduced by language-model-assisted contributor text

Determine whether language-model assistance in authoring commit messages, code reviews, and issue comments causes BurnRiSc’s text-based signals to mask exhaustion through linguistic regularization or instead indicates depleted contributor capacity through delegation.

Background

BurnRiSc uses linguistic features extracted from contributor-authored commits, reviews, and issue comments to approximate the exhaustion and disengagement dimensions of the Oldenburg Burnout Inventory. This operationalization assumes that the observed language reflects the contributor’s own linguistic behavior.

The paper identifies language-model assistance as a potential threat to that assumption. If contributors increasingly use LLMs to generate repository text, BurnRiSc may measure generated language rather than the contributor’s underlying behavioral or linguistic state. The authors explicitly leave unresolved the direction of the resulting bias: language-model mediation might make exhausted contributors appear less exhausted by regularizing their language, whereas delegating communication or coding work to an agent might itself reflect depleted capacity.

References

Beyond these, other questions remain open. Signals computed from contributor-authored text assume a human author. However, contributors may write commit messages, reviews, and issue comments with the help of LLMs. These signals would then measure generated text rather than the contributor's own linguistic behavior. The direction of bias is unclear: mediation may mask exhaustion by regularizing language, or delegation may itself signal depleted capacity.

— BurnRiSc: Toward Non-Invasive Burnout Screening in Open Source from Public Repository Signals  (2609.19422 - Sanko et al., 16 Sep 2026) in Section 5, “Discussion and Further Work,” subsection “Future Plans”