Definition of Hilbert’s avg. pass@ metric on PutnamBench

Determine the precise definition and computation procedure of the "avg. pass@" metric used to report Hilbert’s performance on the PutnamBench benchmark, including how parallelized sub-agent calls are aggregated and how the metric relates to standard pass@n evaluation.

Background

In reporting leaderboard comparisons on PutnamBench, the authors explain that the Compute column lists pass@n for most systems, whereas Hilbert reports an "avg. pass@" reflecting its parallelized agentic setup.

The authors explicitly note that the meaning of this averaged metric is not clearly specified, complicating fair comparison across systems and raising the need to precisely define the metric and its aggregation method.