Attribution of reported ML gains to model design versus data-generation choices

Determine whether the performance improvements reported for machine-learning methods in steady-state transmission grid analysis are attributable to the models’ architectural/design choices or to the dataset generation assumptions (including loads, generator dispatch, and topology).

Background

The paper notes that many prior studies rely on custom, unpublished data pipelines with differing assumptions about loads, dispatch, and topology, which undermines reproducibility and fair comparisons. In this context, the authors explicitly state that it is unclear whether reported gains are due to model design choices or data-generation choices, identifying a concrete unresolved question about attribution of performance improvements.

gridfm-datakit is proposed as a unified, scalable platform to standardize and diversify data generation, potentially enabling more reliable benchmarking and the disentanglement of model effects from data effects. However, the causal attribution of performance gains remains explicitly identified as unclear in the current literature.