Dice Question Streamline Icon: https://streamlinehq.com

Reliability of IMs built from models learned on training data

Investigate the validity and reliability properties of possibilistic inferential models when the ranking and/or validification steps depend on models learned from separate training data, and establish conditions under which strong validity is preserved.

Information Square Streamline Icon: https://streamlinehq.com

Background

Standard IM constructions assume the model form is given. In modern practice, models are often learned from training data and then deployed for inference or prediction on new data.

The concluding open problems highlight uncertainty about the reliability of IMs in this setting, where training-dependent components may affect calibration and validity.

References

There are far too many open problems to list out here, but below are a few that seem particularly interesting, touching on theory, methods, computation, and applications. What about IMs constructed based on models learned from training data?

Possibilistic inferential models: a review (2507.09007 - Martin, 11 Jul 2025) in Section 6 (Conclusion)