Security Incentivization: An Empirical Study of how Micropayments Impact Code Security

Abstract: Security often receives insufficient developer attention because it does not directly generate visible value, leading to underinvestment in practice. We evaluate a countermeasure by team-level incentives tied to measurable security improvements over time. Our semi-automated mechanism aggregates static analysis findings from Bearer, Detekt, and mobsfscan, computes security issue density, and rewards teams based on the relative improvement ratio across sprints, enabling repeatable, scriptable reporting at scale. In a controlled course experiment with 84 students across 14 teams, we compared a security-incentivized condition, in which bonus points were linked to security scanner results, against a control condition with an otherwise identical grading scheme. The treatment group achieved significantly lower security issue density overall (beta regression: $β= -0.396, p = 0.0342$), indicating improved measurable security under incentivization. After controlling for platform, we observed a marked front-end/back-end disparity, with back-ends showing fewer issues and higher improvement ratios under incentives, highlighting heterogeneous effects across stack layers. Notably, these gains were not the byproduct of inflated code volume, as lines of code increased similarly across groups over time. The measurement pipeline and toolchain proved feasible for scripting and automation, supporting scalable adoption in practice. Our results suggest that aligning rewards with automated security metrics can measurably improve code security and merit follow-up in professional contexts and longer development lifecycles.

• The paper shows that team incentives tied to security improvements reduced median SAST issue density from 0.08 to 0.05 issues per line of code, with a significant treatment effect (β = −0.396, p = 0.0342).
• The paper uses a semi-automated pipeline combining Bearer, Detekt, and mobsfscan scans with sprint-based issue-density measurements and team rewards for relative improvement.
• The paper finds that security incentives did not suppress code production but produced uneven benefits, with especially strong improvements in back-end code and important limitations from student participants, SAST proxies, and platform effects.

Overview

This paper reports a controlled empirical study of whether tying team-level rewards to measurable security improvements reduces security defects in software. The authors, motivated by the well-documented underinvestment in security—since security generates no visible value and only prevents damage—design a semi-automated mechanism that aggregates static analysis findings, computes a security issue density metric, and awards bonuses based on relative improvement across sprints. In an experiment with 84 students organized into 14 teams at the University of Klagenfurt, teams incentivized for security achieved significantly lower security issue density than control teams incentivized for general code quality (median 0.05 vs. 0.08 issues per LOC; $\beta = -0.396$, $p = 0.0342$ in beta regression). The paper contributes both the measurement mechanism and the experimental evidence supporting its efficacy.

Background and motivation

The authors ground their work in the economics of information security: security is a cost center that does not add functionality, so developers derive little satisfaction from it and organizations may rationally underinvest, particularly when customers will not pay premiums for more secure products. Prior studies of why vulnerable code is produced identify misunderstanding of security concepts, insufficient resources, and API blind spots as key causes. The rise of AI coding assistants adds urgency, since such tools may amplify insecure code production.

The paper positions itself within incentive-based security research, distinguishing its approach from gamification based on social comparison (which can induce perverse incentives), from risk-transfer mechanisms such as insurance, and from bug bounty programs that reward finding flaws post-deployment rather than preventing them during development. It builds on earlier Shapley-value-based micro-payment schemes for individual developer attribution but deliberately simplifies to team-level rewards: attributing scan findings to individual commits is unreliable due to squashed merges, incomplete commits, and multi-author fixes. Rewards are therefore paid equally to all team members for changes merged into the main branch, trading off the null-player axiom of the Shapley framework against avoiding destructive intra-team competition.

The measurement mechanism

The mechanism follows four steps: scan, estimate change, pay rewards, repeat per sprint. Security issue counts $I_t$ are summed across three SAST tools chosen for scriptability and SARIF-compatible output—Bearer (Java), Detekt (Kotlin), and mobsfscan (Java/Kotlin mobile)—after evaluating and rejecting Snyk, Aikido, HCL AppScan, Semgrep, AppSweep, and SonarQube for poor coverage or integration difficulty. Duplicate findings across scanners are intentionally retained, on the rationale that multiply-detected issues merit prioritization. Security quality is measured as issue density $Q_t = I_t / \text{LOC}_t$, normalizing away code size to avoid penalizing larger codebases, and improvement is captured by the ratio $\Delta Q = Q_{t_{i+1}} / Q_{t_i}$ between consecutive sprints. Rewards $R(\Delta Q)$ are monotonically decreasing in $\Delta Q$: improvements ($\Delta Q < 1$) earn bonuses; deteriorations are not punished.

A pre-study during winter term 2024/25 at Johannes Kepler University Linz validated feasibility: Jenkins pipelines scanned main-branch commits after each push, emailed SARIF-based reports to students, and logged results, with manual spot-checks confirming scanner accuracy. Notably, code-quality findings were included in the issue count on the rationale that poor quality can evolve into vulnerabilities—a construct-validity decision discussed below.

Experiment design

The main experiment embedded the incentive scheme in a software engineering lab course (summer term 2025). Fourteen teams of fourth-semester undergraduates built networked multiplayer Android games over three sprints, each maintaining a Kotlin front-end and a Java or Kotlin back-end. Six teams (32 students) formed the control group (CON), whose bonus points depended on SonarQube code-quality metrics—the bonus system used in prior course iterations. Eight teams (52 students) formed the treatment group (SEC), whose identical bonus (up to 20 points, enough to raise the final grade one step) depended on reducing security issue density via Equation (2) of the paper. Both conditions received both SonarQube and security scan reports at every assessment; only which report was grade-relevant differed, isolating the incentive structure from mere awareness of security monitoring.

The design controlled self-selection (enrollment preceded condition assignment), instructor confounding was partially addressed (one instructor taught sections in both conditions), and researchers not involved in teaching performed scans and analysis. Students were not told they were part of a study; the authors defend this non-disclosure ethically, noting both conditions could earn identical maximum points, bonuses could only improve grades, and the control replicated historical grading practice.

Results

Global effect. SEC teams had significantly lower security issue density than CON teams (medians 0.05 vs. 0.08; beta regression coefficient $\beta = -0.396$, $p = 0.0342$).

Productivity. Poisson regression of lines of code showed LOC grew roughly sixfold from Sprint 1 to Sprint 3, with SEC teams producing somewhat more code growth than CON. This rules out the alternative explanation that security gains came at the cost of reduced output; incentives improved quality without suppressing productivity.

Effort. Absolute security issue counts rose sharply after Sprint 1 (as functionality filled out the architectural skeleton) and remained elevated, particularly in front-ends. SEC teams had significantly fewer absolute issues overall, with a significant group-by-layer interaction: CON teams showed similar issue counts in front-ends and back-ends, whereas SEC teams concentrated their cleaning effort server-side.

Efficiency and layer disparity. After controlling for platform in a beta regression of issue density, the group effect lost significance ($p = 0.0342$0, $p = 0.0342$1); the dominant significant effect was the front-end/back-end disparity itself (front-end coefficient $p = 0.0342$2, $p = 0.0342$3). SEC back-ends reached near-zero issue density after Sprints 2 and 3, while several CON back-ends retained up to 0.28 issues per LOC. Average improvement ratios $p = 0.0342$4 were about twice as large for SEC back-ends (4.91) as for CON back-ends (2.73), though these differences were not statistically significant. The authors interpret this as evidence that incentives aligned effort toward cleaner server-side implementations, while speculating that usability emphasis in front-end work dilutes security salience there—an explanation they flag as speculative and open to further study.

A notable tension runs through the results section: the headline global effect is significant, but the more granular density analysis attributes most variance to stack layer rather than treatment. The honest reading is that the incentive effect is real but heterogeneous and partly confounded with platform, a point the authors state plainly.

Limitations and open questions

The paper is candid about several threats. Internal validity: imperfect crossing of instructors and conditions leaves residual instructor effects; team sizes differed between conditions (6.5 vs. 5.3 average); metric gaming (introducing issues early to inflate later improvement ratios) was not observed but cannot be excluded; no correction for multiple comparisons was applied, so individual significant results warrant caution. Construct validity: SAST findings are proxies, not proof of exploitable vulnerabilities; no penetration testing or dynamic analysis validated actual security impact. Issue density assumes risk scales linearly with code size and ignores severity; summing three scanners without deduplication could inflate counts unpredictably depending on tool overlap. External validity: participants were undergraduates without formal security training, projects were low-stakes games on a single platform at a single institution over one semester, and the reward was grade points rather than money. Whether monetary incentives produce comparable effects among professional developers, and whether gains persist through deployment and maintenance, remain open questions the paper explicitly defers to replication studies.

Conclusion

This paper provides controlled experimental evidence that aligning team rewards with automated security metrics measurably reduces tool-reported security issue density, without inflating code volume, and reveals a pronounced front-end/back-end heterogeneity in how those incentives land. The scanning-and-reward pipeline is scriptable and reproducible, making adoption in industrial settings straightforward in principle. The principal caveats—that the outcome is SAST-reported issues rather than demonstrated security, that the sample is small (14 teams) and drawn from a single student population, and that the granular density effect is dominated by platform rather than treatment—mean the result should be read as promising evidence for the mechanism's efficacy rather than a definitive demonstration, and they define precisely what follow-up work in professional contexts must establish.