Watermarking Game-Playing Agents in Perfect-Information Extensive-Form Games

Abstract: Watermarking techniques for LLMs, which encode hidden information in the output so its source can be verified, have gained significant attention in recent days, thanks to their potential capability to detect accidental or deliberate misuse. Similar challenges involving model misuse also exist in the context of game-playing, such as when detecting the unauthorized use of AI tools in gaming platforms (e.g., cheating in online chess). In this paper, we initiate the study of how game-playing strategies can be watermarked. We show how the KGW watermark for LLMs can be adapted to watermark game-playing agents in perfect-information extensive-form games. The watermark can then be detected using a statistical test. We show that the degradation in the quality of the watermarked strategy profile, quantified by the expected utility, can be bounded, but there is a tradeoff between detectability and quality. In our experiments, we bootstrap the watermarking framework to various chess engines and demonstrate that a) the impact of the watermark on the quality of the strategy is negligible and b) the watermark can be detected with just a handful of games.

• The paper introduces a novel adaptation of the KGW watermarking method for verifying game-playing agents in perfect-information extensive-form games.
• It employs randomized logit-biasing and statistical detection to ensure robust watermarking while maintaining minimal impact on game performance, as validated in chess engine evaluations.
• Empirical results confirm negligible Elo degradation and rapid detectability, fulfilling theoretical bounds on utility loss and quality-detectability tradeoffs.

Watermarking Game-Playing Agents in Perfect-Information Extensive-Form Games: An Expert Summary

Introduction and Motivation

This paper presents a principled approach for watermarking game-playing agents in perfect-information extensive-form games (EFGs) by adapting the KGW watermarking framework, originally developed for LLMs, to the domain of strategic decision-making. The motivation stems from escalating concerns regarding unauthorized usage, model plagiarism, and IP violations in high-performance game-playing AIs, particularly in online chess and similar platforms. Notably, the work aims to provide a verifiable method for detecting agent misuse while formally bounding the utility-loss/quality-detectability tradeoff inherent to watermarking.

Methodological Framework

KGW Watermark Adaptation

The key technical contribution is the adaptation of the KGW watermark's randomized logit-biasing mechanism to extensive-form game strategies. While the core of the KGW watermark in LLMs involves seeding a pseudo-random number generator to partition the token space (vocabulary) into “green” and “red” lists and then boosting logits for “green” tokens by a hardness parameter $\delta$, its adaptation to EFGs replaces tokens with available actions at a game history $h$. Critical modifications include:

• Partitioning Actions: The set of available actions $A(h)$ is probabilistically split into green (size $\gamma|A(h)|$) and red actions.
• Biasing Utilities: For action selection, estimated expected utilities are shifted: $+\delta$ for green, $-\gamma\delta/(1-\gamma)$ for red, keeping the expected bias zero.
• Seeding: Instead of the last action, the hash of the current observation at $h$ is used to seed the PRNG, circumventing replay and splicing-based attacks peculiar to multi-agent interaction.

The watermarking process thus acts as a wrapper around any (possibly pre-trained) game-solving policy, imposing the statistical signature with negligible need for retraining.

Statistical Detection

Watermark presence is detected via statistical testing of deviations in the frequency of green-list actions against the null hypothesis of uniformity. For $n$ actions with $n_G$ on the green list, the detection statistic is:

$$z = \frac{n_G - \gamma n}{\sqrt{n\gamma(1-\gamma)}}$$

A threshold of $h$0 yields a conservative false positive probability ($h$1), aligning with LLM watermarking standards.

Expected Utility Watermarking

Watermark robustness extends to scenarios where users may access multiple action utilities. A complementary watermark modifies displayed expected utilities so that attackers cannot trivially reverse-engineer the watermark by maximizing over unaltered values. When watermark parameters are secret, this process preserves indistinguishability.

Theoretical Analysis

The authors supply rigorous utility-loss bounds under both independence assumptions (Theorem 1) and generality (Proposition 1). For $h$2 actions, the loss $h$3 incurred due to watermarking satisfies:

$h$4

There is an explicit quality-detectability tradeoff: increasing the hardness parameter $h$5 accelerates detection yet degrades expected utility. The analysis leverages standard probabilistic concentration arguments and ties maximal loss to the watermark parameters, allowing informed parameter selection for practical deployments.

Empirical Evaluation

Chess Engine Integration

Comprehensive empirical evaluation is conducted on six mainstream chess engines (including Stockfish 17.1, asmFish 9, Dragon, Ethereal) using the UCI protocol. Watermarked agents were pitted against their originals for 100 games per match, bootstrapping the watermarking mechanism without modification of the underlying evaluation or search code.

Key empirical results:

• Negligible Elo impact: The Elo differences between watermarked and unmodified versions remain within error margins. No engine demonstrates consistent, statistically significant regression; in several cases, the watermarked variant outperforms due to game stochasticity.
• Rapid Detectability: For optimal $h$6, $h$7 settings, z-scores for the detection statistic exceed the $h$8 threshold within 1–9 rounds, well within standard monitoring intervals for online play.
• AUC Metrics: ROC AUC scores for rounds-based detection lie in the $h$9–$A(h)$0 range, indicating robust detectability. Ablation shows that higher $A(h)$1 enables faster or more confident detection at a measurable cost in playing strength.

Parameter Ablation

Ablation studies systematically vary the “green” set proportion $A(h)$2 and hardness $A(h)$3 on Stockfish. Findings indicate $A(h)$4 and moderate $A(h)$5 provide an optimal balance, maximizing detectability with minimal strength degradation. Excessive $A(h)$6 markedly reduces Elo rating.

Discussion: Robustness, Attacks, Broader Applicability

Robustness and Attacks

Watermark robustness is enhanced in the EFG setting versus LLMs due to the irreversible nature of action sequences and multi-agent observability (precluding “offline” removal). Only “online” attacks, such as querying with similar but distinct observations, are plausible. These can be addressed by locality-sensitive hashing and result in minimal advantage due to the risk of strategic error.

Sampling or maximizing over returned expected utilities (for action selection) cannot avoid detection unless utility values are unaltered or the watermark is public, reducing attack surface significantly.

Alternative Watermarks

The logit-biasing/green-list family (KGW and similar) are particularly amenable to this adaptation. Watermarks designed for sampled sequences (e.g., SynthID) fail in EFGs without further smoothing or continuous relaxations.

Application Scope and Extensions

The framework applies cleanly to both competitive and single-agent sequential decision-making. Extending to imperfect-information EFGs, where strategy mixing is essential for optimality, remains non-trivial. Future work is necessary to analyze the impact of watermarking on exploitability and signal robustness in mixed-strategy spaces.

Conclusion

The paper establishes a rigorous foundation for watermarking game-playing strategies in perfect-information EFGs by adapting the KGW mechanism from LLMs. The methodology allows verifiable agent-usage detection with formally bounded utility-loss, validated by empirical evidence of negligible Elo degradation in chess engines and strong statistical detectability after minimal play. The approach is readily bootstrapped to existing solvers and robust against relevant attack vectors. Extensions to imperfect-information games pose a significant, open research challenge with high practical salience.