Validate the F1-spread pre-benchmark indicator across datasets

Test whether the spread of F1 scores across inexpensive random feature subsets predicts when solver and feature-selection-method choices will materially affect downstream performance on additional datasets.

Background

The paper observes that F1 varies roughly ten times more across feature-selection methods on the ULB fraud dataset than on the AmEx default dataset at comparable relative feature budgets. It proposes estimating this spread from inexpensive subsets before running a full solver benchmark, but does not establish whether the relationship generalizes beyond the two datasets studied. Further datasets and random-subset experiments are therefore needed to assess the proposed indicator.

References

Several questions remain open. We have not compared Dirac-3's solutions against Gurobi's certified QUBO energies, which is needed to tell near-degenerate objectives apart from incomplete convergence. Tier 3 on AmEx relies on a classical pre-filter to fit Dirac-3's capacity. Testing F1 spread over random subsets as a pre-benchmark indicator, and extending the comparison to further datasets, would show whether the pattern observed here generalizes.

— Landscape-Dependent Performance of Photonic Quantum Solvers in QUBO Feature Selection for Financial Risk Detection  (2610.03161 - Sahoo et al., 2 Oct 2026) in Section 5.5, Overlap Versus Performance Sensitivity; Section 6, Conclusion