Deduplication Methodology for Repeated-Strategy Submissions

Develop a principled deduplication methodology for analyzing attack strategies when participants submit many attempts that consistently follow a single strategy, in order to mitigate bias in distribution and success-rate estimates within the Indirect Prompt Injection Arena dataset.

Background

Participants could iterate rapidly and often submitted many similar attempts, sometimes adhering to a single strategy across numerous trials. This can bias the observed distribution of strategies and their apparent success rates.

The authors indicate that the appropriate way to deduplicate such submissions is unclear, leaving a methodological gap for future work to address.