Dice Question Streamline Icon: https://streamlinehq.com

Identifying a valid instrument to address endogeneity in the Stockfish setting

Identify and validate an instrumental variable Z for the computer chess (Stockfish) setting that affects the instantaneous efficiency growth rate \dot A/A only through its influence on input intensity I(t) (e.g., Fishtest test counts), with sufficient data availability to implement instrumental-variable estimation of the causal parameter λ.

Information Square Streamline Icon: https://streamlinehq.com

Background

The inputs used to model software-efficiency progress in Stockfish (tests completed on Fishtest) may be endogenous, potentially responding to perceived research promise, which biases causal estimates of the input elasticity λ.

The standard econometric remedy is an instrumental variable that influences \dot A/A solely via I(t). However, the authors report being unable to find a viable instrument that both satisfies the exclusion restriction and has sufficient data coverage.

Establishing a credible instrument would enable causal identification of λ and strengthen conclusions about the influence of inputs on software progress.

References

The most common way to control for endogeneity in econometrics is to use an instrument: that is, find some variable Z that we expect to causally influence \dot A/A only through its influence on I. Finding such an instrument is tricky, however, and we have not managed to find a Z for which we both have sufficiently abundant data and for which the associated causality assumption seems significantly more plausible than assuming strict exogeneity of the inputs themselves.

Estimating Idea Production: A Methodological Survey (2405.10494 - Erdil et al., 17 May 2024) in Section 6.2, Computer chess — Endogeneity problems