Identifying which Seed baseline query was used
Ascertain which of the two available Boolean queries in the Seed dataset—the “query” or the “edited-search”—was used as the baseline in prior evaluations, and standardize documentation to enable reproducibility of baseline comparisons.
References
However, since the Seed dataset also contains a second Boolean query edited-search, we also show the results for this query (Baseline-edit in the result tables), as it is unclear which query has been used based on the documentation.
— A Reproducibility and Generalizability Study of Large Language Models for Query Generation
(2411.14914 - Staudinger et al., 22 Nov 2024) in Section 4.1 Reproducibility and Generalizability Study