Conduct a broader multi-seed comparison of C-MoA and CONTRA-MoA

Compare C-MoA and CONTRA-MoA across multiple random seeds and experimental splits to determine whether the reported negative counterfactual result on the single frozen FactScore split is robust.

Background

The paper’s counterfactual comparison is based on one frozen FactScore split. On that split, adding the counterfactual and stability signals through CONTRA-MoA lowered the AUC relative to the agreement-only C-MoA score, because the verifier was often unable to distinguish true claims from plausible near-misses for obscure entities.

A multi-seed comparison would assess the stability and generality of this negative result across alternative random partitions and repeated experimental runs. The paper explicitly identifies this broader evaluation as unresolved rather than presenting it as an established limitation that has already been settled.