Effect of substantially larger training sets on NRE overconfidence

Determine whether increasing the neural ratio estimation training set by an order of magnitude or more substantially reduces the overconfidence of the inferred posteriors.

Background

The authors train neural ratio estimation models on approximately 500,000, 1 million, 1.5 million, and 2 million simulated lens images. Increasing the training set by a factor of four produces no detectable reduction in posterior overconfidence, leaving open whether much larger training sets could improve calibration. Resolving this question would clarify whether the observed overconfidence is primarily a finite-training-sample effect or an intrinsic limitation of the approach and architecture used.

References

For the labeled extension, studying calibration under progressively smaller labeled sets would clarify its sample efficiency and establish how much supervision is sufficient for reliable anchor refinement.

— When Agents Disagree: Bayesian Backward Reasoning as a Label-Free Anchor for Multi-Agent Collective Decision-Making  (2609.11709 - Chen et al., 10 Sep 2026) in Section Limitations

Increasing the training sample by a factor of four produces no detectable reduction in overconfidence, though we cannot rule out that increases of an order of magnitude or more may be required to see a significant effect.

— Strong Lensing Cosmology with Population-level Calibrated Neural Ratio Estimation  (2608.23534 - Jarugula et al., 24 Aug 2026) in Section 6.1, Effect of training set size